Spectraforce Technologies Hiring for Data Engineer Lead at Newark, NJ  Full Time

Responsibilities:
Implement and support:

• end-to-end data lake/warehousing/mart/business intelligence/ analytics/ services solutions (ingest, storage, integration, processing, services, access) in AWS.
• data lake data intake/request/onboarding services and service documentation
• data lake ingestion services for batch/real time data ingest and service documentation.
• data processing services (ETL/ELT) for batch/real time (Glue/Kinesis/EMR) and service documentation
• data storage services for data lake (S3)/ data warehouses (RDS/Redshift)/ data marts and service documentation
• data services layer including Athena, Redshift, RDS, microservices and APIs.
• pipeline orchestration services including lambda, step functions, MWAA (optional)
• data security services (IAM/KMS/SM/encryption/anonymization/RBAC) and service documentation
• data access provisioning services (Accounts, IAM Roles RBAC), processes, documentation, and education
• data provisioning services for data consumption patterns including microservices, APIs and extracts.
• metadata capture and Catalog services for data lake(S3/Athena), data warehouses (RDS/Redshift), Microservices/APIs
• metadata capture and Catalog services for pipeline/log data for monitoring /support
• Implement CI/CD pipelines .
• Prepare documentation for data projects utilizing AWS based enterprise data platform.
• Implement high velocity streaming solutions using Amazon Kinesis, SQS, and SMS
• Migrate data from traditional relational database systems to AWS relational databases such as Amazon RDS, Aurora, and Redshift
• Migrate data from traditional file systems and NAS shares to AWS data lake (S3) and relational databases such as Amazon RDS, Aurora, and Redshift
• Migrate data from APIs to AWS data lake (S3) and relational databases such as Amazon RDS, Aurora, and Redshift
• Provide cost/spend monitoring reporting for AWS based data platform data initiatives.
• Provide governance/audit reporting for access of AWS based data platform.
• Lead the implementation of a data lake strategy to enable LOBs and Corporate Functions with a robust, holistic view of data – driven decision making.
• Serve as delivery lead for EDP data initiatives product owner.
• Partner with immediate engineering team, product owner, IT, partners on EDP agenda
• Provide technology thought leadership, consulting, and coaching/mentoring.
• Establish development, QA, stage, and production migration/support processes.
• Establish best practices for development and support teams.
• Deliver end-end data initiatives from ingest-consume via microservices/ APIs, JDBC/ ODBC, file extracts etc.
• Work with scrum master to develop and own backlog, stories, epics, sprints.

Qualifications

• Bachelor’s degree in computer science, Software Engineering, MIS or equivalent combination of education and experience
• Experience implementing, supporting data lakes, data warehouses and data applications on AWS for large enterprises.
• Programming experience with Java, Python/ Scala, Shell scripting
• Solid experience of AWS services such as CloudFormation, S3, Glue, EMR/ Spark, RDS, Redshift, DynamoDB, Lambda, Step Functions, IAM, KMS, SM etc.
• Solid experience implementing solutions on AWS based data lakes.
• Experience implementing metadata solutions leveraging AWS non-relational data solutions such as ElastiCache and DynamoDB
• AWS Solutions Architect or AWS Big Data Certification preferred.

• Experience in AWS data lake/data warehouse/business analytics.
• Experience and understanding of various core AWS services such as IAM, Cloud Formation, EC2, S3, EMR/Spark, Glue, Datasync, CloudHealth, CloudWatch, Lambda, Athena, and Redshift
• Experience in system analysis, design, development, and implementation of data ingestion pipeline in AWS.
• Experience with DevOps and Continuous Integration/ Delivery (CI/ CD) concepts and tools.
• Experience with business intelligence tools such as Tableau, Power BI or equivalent.
• Knowledge of ETL/ ELT
• Experience in production support from Level 1 to Level 3
• Awareness of Data Management & Governance tools
• Working experience with Hadoop, HDFS, SQOOP, Hive, Python, and Spark is desired.
• Experience working on Agile projects.

Below is the brief introduction of SPECTRAFORCE, the benefits we offer and the disclaimers:
Established in 2004, SPECTRAFORCE is one of the largest and fastest-growing diversity-owned staffing firms in the US. The growth of our company is a direct result of our global client service delivery model that is powered by our state-of-the-art A.I. proprietary talent acquisition platform, robust ISO 9001:2015/ISO 27001 certified processes, and strong and passionate client engaged teams.
We have built our business by providing talent and project-based solutions, including Contingent, Permanent, and Statement of Work (SOW) services to over 130 clients in the US, Canada, Puerto Rico, Costa Rica, and India. Key industries that we service include Technology, Financial Services, Life Sciences, Healthcare, Telecom, Retail, Utilities and Transportation.
SPECTRAFORCE is built on a concept of “human connection,” defined by our branding attitude of NEWJOBPHORIA, which is the excitement of bringing joy and freedom to the work lifestyle so our people (and clients) can reach their highest potential. Our entire workflow and teams are trained to cultivate the joy of NEWJOBPHORIA with candidates and employees throughout their engagement with SPECTRAFORCE. http://www.spectraforce.com
Benefits: SPECTRAFORCE offers ACA compliant health benefits as well as dental, vision, accident, and hospital indemnity insurances. Additional benefits SPECTRAFORCE offers to the eligible employees include commuter benefits, 401K plan with matching and a referral bonus program. SPECTRAFORCE offers unpaid leave as well as paid sick leave when required by law.
Equal Opportunity Employer: SPECTRAFORCE is an equal opportunity employer and does not discriminate against any employee or applicant for employment because of race, religion, color, sex, national origin, age, sexual orientation, gender identity, genetic information, disability or veteran status, or any other category protected by applicable federal, state, or local laws. Please contact Human Resources at [email protected] if you require reasonable accommodation.