Data Infrastructure Engineer (Big Data)

Big Data Processing:

Implement and manage big data processing systems. Experience or strong interest in big data implementation is required.Data Pipeline Implementation:

Develop and maintain robust data processing pipelines. Candidates should have experience or a strong interest in data pipeline architectures.Batch vs. Real-Time Processing:

Clearly articulate the differences in implementation strategies between batch processing and real-time APIs.Streaming Data Responsibilities:

Explain the separation of responsibilities between producers and consumers in streaming data processes.Database Management:

Build and operate databases storing over 10 GiB of data, ensuring efficiency and scalability.Data Platform Operations:

Operate platforms such as Amazon Redshift, Google BigQuery, Snowflake, and Databricks. Experience in managing these or similar platforms is highly desirable.Qualifications and Skills

Experience with Big Data:

Proven track record in handling large-scale data projects, with specific skills in time-series databases, streaming data processing, and multi-tiered database architectures.Data Warehousing and Data Lakes:

Hands-on experience with data warehouse and data lake technologies, including understanding of Lambda architecture.Technical Proficiency:

Strong technical skills in relevant big data technologies and frameworks.Problem Solving:

Excellent analytical and problem-solving skills, capable of managing complex data challenges.Communication:

Effective communication skills, able to document and explain data processes clearly to both technical and non-technical stakeholders.Nice to Have

Experience with cloud platforms such as AWS, Google Cloud Platform (GCP), or Microsoft Azure.Knowledge of cloud-based data storage solutions (e.g., S3, Google Cloud Storage, Azure Blob Storage).Familiarity with cloud-based data processing services (e.g., AWS Lambda, Google Cloud Dataflow, Azure Data Factory).Experience with cloud infrastructure automation and management tools (e.g., Terraform, CloudFormation, Ansible).Machine Learning Integration:

Understanding of integrating machine learning models into data pipelines.DevOps Practices:

Experience with DevOps practices and tools for continuous integration and deployment (CI/CD).Data Security:

Knowledge of data security best practices and compliance standards in cloud environments.Visualization Tools:

Experience with data visualization tools and platforms (e.g., Tableau, Power BI, Looker).Programming Languages:

Proficiency in additional programming languages relevant to data processing and backend development (e.g., Scala, Go, Rust).Apply for this job

*indicates a required fieldFirst Name *Last Name *Email *Phone *Resume/CV *Enter manuallyAccepted file types: pdf, doc, docx, txt, rtfEnter manuallyAccepted file types: pdf, doc, docx, txt, rtfLinkedIn Profile (If you don't have a profile, please type N/A)

*Telegram username (If you don't have a profile, please type N/A)

*Twitter username (If you don't have a profile, please type N/A)

*WebsiteWhat are your salary expectations? *What country and time zone do you reside in? *When are you available to start? *

Select...Are you legally authorized to work in the country in which you are currently domiciled? *

Select...Do you now, or will you in the future, require sponsorship for employment work permit or visa status to work legally for our Company?

Select...Are you legally authorized to work in the United States?

Select...How many years have you worked with big data technologies? *

Select...How do you rate your experience with streaming data platforms? *

Select...What is the largest database (in terms of data volume) you have managed? *

Select...Do you have experience with Lambda architecture in data projects? *

Select...Which of the following big data processing systems have you implemented or managed? Managed Service is also possible. (Please select all that apply)

*- Hadoop- Spark- Flink- Other (Please specify)Have you developed data pipelines? If so, which tools have you used? (Select all that apply) *- Other (Please specify)Which of the following data platforms have you operated? (Select all that apply)

*

#J-18808-Ljbffr