Darren Rabbitt

Python development on AWS for a client in financial services.

• Data pipelines and the API layer of the platform, in Python on AWS.

Key skills

• Python
• AWS
• APIs
• Data pipelines
• AI coding agents

Architected and delivered an enterprise-wide automation framework validating data pipelines and workflow orchestration on Cloudera on AWS, in Kerberos-secured environments.

• A Python/BDD framework that let QA and non-technical teams validate pipelines through human-readable workflows.
• Regression suites that validated major platform and cluster upgrades: HDFS, YARN, Spark, Hive, Phoenix and Oozie.
• Automated end-to-end validation of Spark runs, S3 file movement, schemas, rules and transformations.
• Led a test migration between orchestration platforms with zero workflow disruption, and handled SIT release sign-off through automated regression packs.

RelatedBDD framework design guidepytest-allure-host

Key skills

• Python
• pytest
• BDD
• Allure
• Pydantic
• Spark
• Hive
• HBase
• YARN
• Airflow
• Oozie
• Kerberos
• AWS
• Jenkins
• Docker
• CI/CD

Senior Software Engineer from Oct 2021; promoted to Staff in Feb 2024.

Platform engineering: developer experience, Kubernetes fleet tooling, AWS cost and CI/CD reliability across Lacework's microservices.

• Automated retirement of under-used Kubernetes environments (usage analysis, health checks, Terraform cleanup), reducing AWS spend by $600k+ a year. It became the standard process for retiring dev and QA clusters.
• Built the support-matrix automation (Python, Terraform, CI) that certified the product's agent across AMIs, Docker versions, Kubernetes distros and EKS, cutting turnaround from weeks to days.
• Moved teams to local microservice development with Tilt, and onto a unified Codefresh CI pipeline.
• Bazel/Starlark tooling to analyse service adoption, configuration drift and migration readiness.
• Ran weekly code screens for engineering candidates, assessing technical and cultural fit.

RelatedCluster decommissioning guideSupport matrix design guide

Key skills

• Python
• Go
• AWS
• Terraform
• Kubernetes / EKS
• ArgoCD
• GitOps
• Tilt
• Docker
• Bazel
• Codefresh
• GitHub Actions
• Prometheus
• Grafana
• CI/CD

Full-stack development, development in test and platform engineering across D&B's data pipelines and infrastructure.

• A full-stack Spring application on Elastic Beanstalk that served mock data from RDS in place of legacy systems, shortening regression cycles.
• Angular development on an internal API testing tool.
• An ETL validation framework on EMR (Spring, MapReduce) for large data comparisons, 250,000+ records per run.
• Reusable Terraform modules for EMR clusters, and stabilised SaltStack provisioning.
• A Jenkins shared library (Java, Groovy) for provisioning checks, repository tagging and consistent CI across teams.
• Mentored engineers and interns on the team.

Key skills

• Java
• Python
• Groovy
• AWS
• EMR
• Lambda
• Terraform
• SaltStack
• Jenkins
• CI/CD
• HBase
• Spring
• Node.js
• Angular

Led the automation strategy for Integra, Capita's financial product: CI-enabled test pipelines and cross-platform deployment.

• A parallelised web automation framework on a Docker-based Selenium Grid, running 200+ tests against both deployments.

Key skills

• Java
• Docker
• Jenkins
• Groovy
• CI/CD

Automation and Java development in IBM's SmartCloud and Connections teams.

• Built the Connections regression suite in Java, Selenium and TestNG, and the Jenkins and uDeploy CI that ran it.
• Built Auditor checks that validated cloud configuration across IBM data centres in the US, Europe and China.

Key skills

• Java
• Jenkins
• CI/CD