Senior Data Engineer
Home. It’s a word that holds a special place at Bright MLS. At its core, it’s shelter. But it’s also so much more. Family. Community. Safety. A place where you can be your fullest, truest self. That’s the word that inspires all of us at Bright to do the work that we do -- Ensuring an open, clear, and competitive housing market for ALL.
Our company –and our brand –are reflective of the diverse communities that make up our market. Our employees represent a diverse mix of backgrounds, cultures and experiences, so much so that Bright’s been named as one of the most diverse employers in the area by the Washington Business Journal –2 years in a row.
Bright MLS is the engine that powers the real estate market in the Mid-Atlantic U.S. - supporting over $100B in transactions yearly. It is the single source for all data on Mid-Atlantic residential real estate - anywhere. As a Multiple Listing Service (MLS), our technology solutions connect real estate professionals with other real estate professionals and their clients, providing an open and accessible marketplace for buying and selling real estate.
We’re redefining what it means to be an MLS, and we’d love to have YOU here with us helping tell a Brighter story to the world. To learn more, please visit www.brightmls.com.
Overview:
Bright MLS leads the real estate industry in technology and data innovation. Born of the need to help Agents better market homes and identify prospects we continue to evolve what is means to be an MLS and premiere real estate technology provider. Arming our subscribers with the highest quality, most transactable data and products is what we do. Data is at the foundation of our success and the talent within our Data Engineering team is the secret sauce to our winning strategy. Data Engineering builds data and analytics solutions for various use cases including API’s, stream processing, reporting, product analytics, machine learning/AI, marketing optimization and financial reporting. By implementing cutting edge data solutions on our data mesh foundation, we deliver data structures, and insights that are the foundation for real estate transactions and decision-making for our Bright MLS subscribers.
Responsibilities:
- Design and develop efficient and scalable data pipelines between enterprise transactional systems, third-party and analytics platforms
- Must be a crack coder in one of the following (node.js or python)
- Must be proficient in writing and refactoring efficient SQL queries.
- Must be able to explain features of good data model design
- Build and maintain a data environment for speed, accuracy, consistency and ‘up’ time
- Support analytics and data science by building a world-class data mesh environment that empowers analysts to determine insights into revenue and power products across the organization
- Integrate third-party data sources and API’s into the Bright data mesh ecosystem
- Work closely with Data Science team and participate in development of feature engineering pipelines
- Design and develop data products with modern AWS cloud technologies such as S3, Redshift, EMR, Hive, Presto, Flink and Spark
- Work with the machine learning engineering team to build a data eco system that supports AI products at scale
- Design and deploy an enterprise data warehouse that supports internal and market facing analytics products at scale
- Ensure data governance principles adopted, data quality checks and data lineage implemented in each hop of the data
- Partner with adjacent organizations to ensure proper integration and adherence to standards
- Be in tune with emerging trends in data management and cloud technologies and participate in evaluation of new technologies
- Ensure compliance through the adoption of enterprise standards and promotion of best practice / guiding principles aligned with organization standard
Required Skills/Education:
- 8+ years of experience as data engineer at an innovative organization
- 4+ years of hands-on experience in implementing data lake systems using AWS cloud technologies such as S3, Redshift, EMR, Hive, Kafka and Spark
- Expert managing AWS services (EC2, S3, Route 53, ELB, VPC, cloudwatch, Lambda) in a multi account production environment
- Experience with development frameworks as well as data and integration technologies such as Informatica, Python, Scala
- Create new ETLs in AWS Glue with Python or Node.Js as the scripting language
- Create AWS Lambdas using Python or Node.Js as the scripting language
- Modify existing ETLs to fix issues where approach is appropriate
- Use Glue for ETLs inside of AWS to and from all AWS types of data sources
- Support the migration of data into S3, Redshift, DynamoDB, AWS RDS
- Experience With Machine Learning Libraries and Frameworks (TensorFlow, MLlib) is an added advantage
- Exposure to R, SparklyR, and Other R packages is a Plus
- Expert knowledge of Agile approaches to software development and able to put key Agile principles into practice to deliver solutions incrementally.
- Monitors industry trends and directions; develops and presents substantive technical recommendations to senior management
- Excellent analytical thinking, interpersonal, oral and written communication skills with strong ability to influence both IT and business partners
- Ability to prioritize and manage work to critical project timelines in a fast-paced environment
- Advanced knowledge for Microsoft SQL Server for future migration to an AWS Database Platform
Preferred Skills/Education:
- Previous experience with cloud development (AWS, GCP)
- Previous experience in design and deployment of data lakes, data mesh, data warehouse and streaming platforms
- Previous experience with data quality projects and public records
- Previous experience with: AWS DynamoDB, AWS Elastic Map Reduce, AWS Lambda, AWS Step Functions, AWS Redshift, AWS RDS, Terraform or CloudFormation
- AWS Architect Certification is a plus
Additional Notes/Comments:
- BS or MS degree in Computer Science or Information Technology or equivalent experience
It is the company's policy to recruit, hire, train and promote individuals, as well as to administer any and all personnel actions, without regard to race, religion, age, sex, marital status, sexual orientation, disability, national origin, ancestry, military status or any other unlawfully prohibited characteristic in accordance with applicable laws.
