Sr. Software Eng. - Massive Graph
- San Francisco, CA
Job Posted: Thursday, February 22, 2018
Expires On: Tuesday, May 29, 2018
Job Type: Permanent
Duration: 1+ years
Desired Experience: 3-5 years
Work Authorization: Resident/Citizen, Citizen
us process a trillion edge graph as quickly and efficiently as possible. Were LiveRamp
Identity Engineering, and we maintain a massive graph that connects
together the different identifiers for consumers (e.g., anonymized email
addresses and phone numbers) and the devices they use online.
engineering systems weve developed are constantly ingesting new edges from
thousands of different sources and finding numerous types of relevant paths to
power LiveRamps suite of core products.
to work on projects such as:
Path Computer: This system finds relevant graph paths using the pregel graph computation framework as
implemented in Apache
Giraph. There are challenges in running Giraph at the scale of our
graph and were constantly looking to refine our Pregel algorithms.
Ingestion and Partitioning Framework: We could never process all trillion edges
at once and luckily we dont have to. Instead we process subgraphs that contain
specific types of edges. Our edge ingestion and partitioning framework manages
different Hadoop datastores for different types of edges and automates the
ingestion of new edge data. It leverages LiveRamps Seek MSJ framework to efficiently
incorporate new data into existing edge stores.
Computation as a Service: We provide a service to other LiveRamp engineering
teams for finding specific types of paths within our massive graph. It handles
20,000 requests a day and this is possible due to its use of caching and
intelligently batching similar request together.
take pride in operating as a high performance team, while maintaining kindness
We find feedback to be
important in helping us grow as individuals and as a team and were always
looking for chances to share positive and constructive feedback.
We develop in Java and
use MapReduce, Giraph, and Spark. Were open to new technologies and languages
if they help us better solve a problem.
Is currently use a 79,800
core Hadoop cluster with 90 PB of disk space and 256 TB of RAM (shared across
all of LiveRamp data engineering) to power our systems. Were also exploring
moving everything to AWS.
3+ years of experience writing and deploying production code.
a passion for building large scale, distributed systems and are comfortable
writing high performance code.
love mentoring junior engineers, and deploying best practices.
a startup personality: smart, ethical, friendly, hard-working and productive.
a data enthusiast who wants to be surrounded by brilliant teammates and huge
communication and presentation skills
Work with talented, collaborative, and friendly people who love what they do.
Enjoy catered meals, boundless snacks, and the occasional food truck.
We host events such as game nights, happy hours, camping trips, and sports
Every employee is a stakeholder in our future. Health and Saving. Receive the
benefits of comprehensive health, dental, vision and disability insurance along
with a 401k matching plan.
Work in the heart of San Francisco and take advantage of our commuter benefits.
More about us:
is the leader in data connectivity, helping the worlds largest brands use
their data to improve customer interactions on any channel and device. We
thrive on mind-bending technical challenges and value entrepreneurship,
humility, and constant personal growth.
is so much more that we want to build and that we could continue to improve. We
value strong engineers who are agile enough to hit the ground running and
all recruitment agencies: LiveRamp does not accept agency resumes. Please do
not forward resumes to our jobs alias, LiveRamp employees or any other company
location. LiveRamp is not responsible for any fees related to unsolicited
LiveRamp is an affirmative
action and equal opportunity employer (AA/EOE/W/M/Vet/Disabled) and does not
discriminate in recruiting, hiring, training, promotion or other employment of
associates or the awarding of subcontracts because of a person's race, color,
sex, age, religion, national origin, protected veteran, disability, sexual
orientation, gender identity, genetics or other protected status. Qualified
applicants with arrest and conviction records will be considered for the
position in accordance with the San Francisco Fair Chance Ordinance.