Aaron Ball
* **email**: [aaronb@oper.io](mailto:aaronb@oper.io) * **phone**: +1 720-319-7799 * **www**: [https://oper.io](https://oper.io) * **code**: [https://src.oper.io/nullspoon](https://src.oper.io/nullspoon)
PROFILE ======= * **Primary Languages**: Bash (13 years), C (4 years), Python (3 years), CSS and HTML (10 years) * **Tools**: Git (10 years), Jenkins (8 years), Kubernetes (5 years), Ansible (7 years), Terraform (3 years) * **Linux**: 15 years - Crux, Ubuntu, Debian, CentOS, RedHat, Arch, Gentoo I am proficient in infrastructure and programming to improve financial, personnel, time, infrastructure, testing, and software efficiency. I use object oriented theory and Unix design philosophy to create small, simple, maintainable, secure, and efficient automation. The radius of impact of my work can scale from single tasks and programs to large cross-team initiative coordination in an organization.
EMPLOYMENT ==========
DataRobot ---------
* **December 2018 - present** * Technical Lead Manager (2023 - present) * Principal Software Engineer (2022 - present) * Senior Software Engineer (2020 - 2022) * Software Engineer (2018 - 2020)
* **Primary operating systems**: Ubuntu Server (16.04, 18.04, 20.04, and 22.04) * **Primary languages**: Bash, C, Groovy (Jenkins), Makefile, and Python ### Technical Lead Manager * Took on technical and team lead responsibilities. This included high level project management and collaboration with other team leads, weekly one on ones with team members, individual goal planning, weekly organizational tech spec reviews, team priority management, and organization-wide communication. * Participated in engineering technical specification review committee to peer review designs of new features to ensure quality, security, stability, maintainability, and cost effectiveness. * Managed a highly effective team of four engineers. Lead them through several major organizational direction changes, getting all four promoted. Worked with each person on the team to set quarterly growth goals and met weekly with each person for one-on-ones. * Coordinated four very large migration projects affecting the entire engineering organization from old to new technologies, requiring coordinating migration work across an organization of 300 engineers. ### Principle Software Engineer * Built self-maintaining internal LetsEncrypt (ACME) service to generate and maintain signed TLS certs for development infrastructure. Service was internally stateless, backing state to AWS secretsmanager for easy management and automated distribution of newly issued certificates. * Designed, documented, and implemented a Kubernetes-native replacement for AWS NLBs, reducing cost sigifnicantly and improving connection stability for an internal service which served over 25 terabytes of data per day for CI and CD pipelines. * Solved problem of pivoting to Kubernetes native developement and product release for developers and customers by outlining and implementing architecture for migration to a public/private Docker registry with a promotion process for customer artifacts. Built automation to define the public registry state in code (ACLs, repos, organization membership, etc) for a more controllable and auditable setup. * `[Bash]` Completely eliminated cross availability zone traffic costs for Jenkins and Artifactory by building a simple solution which intelligently pinned traffic to the Artifactory cluster instance within the local availability zone. * Worked on AWS cost reduction project. Total contributions yielded $350,000 per year in reduction across the organization without negatively impacting daily operations. Accomplished this with resource cleanup automation, runtime storage and memory compression, instance usage analysis and right-sizing, and removal of cross availability zone costs by intelligently pinning traffic to the local AZ within automation. * Served as interviewer for several teams around the company, running interviews for competencies such as DevOps, SRE, architecture, programming, and culture fit. ### Senior Software Engineer * `[HAProxy]` Built a Kubernetes native ECR proxy service which allowed greater flexibility and ease of use for developers needing to interact with cumbersome ECR endpoints and authentication. Service supported zero-downtime updates. * `[Bash]` Built on-premises virtualization platform of over 230 KVM guest nodes to lower costs in AWS. Clusters were completely autonomous and self-maintaining. Automation registered nodes with their configured Jenkins masters on boot, and deregistered, refreshed to a clean disk state, and reregistered after a Jenkins job finished running, automatically maintaining a clean system state for all jobs with no plugins required and 4 simple scripts. * `[Kubernetes]` Migrated a three node Artifactory cluster from EC2 to Kubernetes using Kustomize to manage and automate multiple environments for testing and release. Maintenance now occur at any time during the day with no downtime due to custom connection draining automation. * `[Bash]` Automated auditing of GitHub traffic to determine opportunities for cost reduction and worked with developers to reduce those costs through on-system caching, sparse clones, on-prem LFS hosting. * `[Kubernetes]` Migrated all Terraform Kubernetes code away from Terraform and into Kubernetes native to improve reliability and state management. ### Software Engineer * Worked on Development Infrastructure team automating processes and infrastructure to support development, testing, engineering productivity, security, cloud operations, and financial operations teams. * Defined upgrade process and worked with all concerned teams to upgrade automation infrastructure from Ubuntu 16.04 to 18.04. Updated all automation and testing code to be compatible with both versions. Performed the same task to migrate from Ubuntu 18.04 to 20.04 one year later. * `[Jenkins]` Version controlled thousands of Jenkins job configurations and enforced peer review by storing all job configs in git repos as JJB (Jenkins Job Builder) templates and deploying with automation on merge request review and approval. * `[Python]` Automated Jenkins credential store management and synchronization across thirteen Jenkins masters using Jenkins Groovy API. * `[Terraform]` Automated configuration management of infrastructure such as EC2, ELB/ALB, EKS, Route 53, RDS, and S3. * Worked with Jenkins, Ansible, Packer, and Terraform to automate system build processes (AWS AMIs, KVM backing disks, laptop provisioning) Jeppesen, A Boeing Company --------------------------
* _(Contract)_ **August 2017 - October 2018** * Software Integration Engineer
* **Primary operating systems**: Centos 6.9 - 7.7, RHEL 6 - 7 * Worked on the environments team supporting about 80 developers * `[Bash]` Developed portable self-extracting installer to integrate and automate the deployment process. The installer reduced a 40 page install document to a single command using a bash header with a multi-stage binary payload footer. * `[Ansible 2.6]` Automated Linux environment setup and management. Wrote roles for deploying and configuring Apache tomcat, Apache httpd, Apache Artemis, Oracle JRE, program-developed software, etc. * Automated software version inventory process to ensure the versions of all pertinent external endpoints in every environment matched as the product advanced through the phases of testing. * `[Jenkins]` Automated deployments to sixteen environments. Deployments triggered by schedules, Git repo tag creation, and commit pushes (each environment with different criteria). * `[Ansible]` Server configuration automation. Wrote scripts to generate dynamic playbooks and dynamic inventories. Roles were separated into two tiers: the vendor tier and proprietary tier for separation of concerns. * Built and administered Okta authentication backend (users, groups, signon policies, service accounts, etc.), for twenty different application instances. * Assisted in planning and automation of migration from on-site datacenter to Microsoft Azure. Truven Health Analytics -----------------------
* **January 2016 - August 2017** * Senior Development Operations Engineer
* **Primary operating systems**: Centos 5.9 - 6.7, RHEL 5 - 6 * Worked as a liaison between devops, configuration management, development, production operations, and application operations teams * `[Bash]` Created and documented scripting libraries to improve builds, deployments, and cluster switching processes. This reduced feature development and bug fix time and increased process scalability, visibility, and stability. * `[Bash]` Implemented a configurable self-extracting package container which enabled complex scripts, frameworks, and resources to be reduced to a single file and subsequently easily deployed on air-gapped systems. * `[Bash]` Wrote a configurable server validation scripting framework for ensuring any given server met the team's deployment requirements for the specified server role (eg: frontend, database, storage, tomcat, etc). * `[Bash]` Automated developer machine build process so developers would have consistent environments in which to write, build, test, and deploy their code. This reduced a 2 week per-employee setup time to about 15 minutes. * `[Perl]` Created a command line Artifactory api wrapper for managing artifacts. Wrapper could download, upload, and copy artifacts to alternate repos (eg: for a promotion process). * `[Ansible]` Wrote roles to deploy and configure MongoDB, Zabbix, Graylog, Elasticsearch, Rsyslog, Tomcat, and others. * Automated migration of more than 3200 SVN repos to Git. * Established and documented Git development and release process standards. * Managed deployments using a combination of Ansible, Bash, Perl, and Jenkins. * Reduced Jenkins-triggered infrastructure deployment process from over 300 jobs down to five jobs. Time Warner Cable -----------------
* **March 2015 - January 2016** * Senior DevOps Engineer
* **Primary operating systems**: Centos 6.5 and 6.6 * `[Bash]` Automated upgrades of the GitHub Enterprise cluster with minimal human interraction, including several several workarounds to unfixed bugs in the appliance software. * `[C]` Wrote parallel processing program to benchmark legacy VMWare network against new OpenStack network * `[C]` Automated audit of 8 million firewall rules and NGINX access logs to determine how many rules were actually needed * `[Puppet 3.2]` Configuration management, using r10k for dynamic environment deployments and Hieradata to support separation of data and modules * Automating legacy VMWare infrastructure (54 hypervisors) for migration to OpenStack CenturyLink, Inc. -----------------
* _(Contract)_ **August 2014 - February 2015** * Linux Engineer
* **Primary operating systems**: CentOS (6.5, 6.6, 7), Ubuntu Server * Deployed and managed Mapr (Hadoop distribution) for development and test clusters * `[Bash]` Automated management and synchronization of users and their home directories, detect and allocate unique uid/gid pairs, and perform various user-related checks across a cluster of any size * `[HAProxy]` Proxy of LDAPS traffic from corporate into the lab (dev/test) network for authentication * MediaWiki administrative tasks (backups, page management, writing templates, etc). * Established team version control best practices within Git * `[Ansible]` Infrastructure configuration automation Intelligent Software Solutions, Inc. ------------------------------------
* _(Contract)_ **March 2014 - August 2014** * DevOps Engineer
* **Primary Operating Systems**: RedHat Linux (6 and 7), Windows 7 * Wrote Chef v10 cookbooks to deploy any combination of war files on top of base tomcat instances * Wrote Chef cookbook compatibility layer for Windows-specific deployments using cookbooks designed for Linux * Built a templatized virtual Chef development environment on Linux KVM with startup times under 5 seconds. DaVita ------
* **October 2012 - March 2014** * Linux/Unix Systems Engineer
* **Primary operating systems**: AIX (5.3 and 6.1) and CentOS Linux (4 - 6) * Standardized, consolidated, and converted team documentation from Sharepoint to MediaWiki * Audited and remediated security holes to harden Linux and AIX servers * `[Perl]` Automated reocurring audit of local and Active Directory users with access to a public facing company FTP site. * `[Perl]` Automated monthly SOX compliance report for the intersection of our global sudoers configs with Active Directory integration - all users and all commands everywhere. * `[Bash]` Automated jump server SSH key rotation every three months and when an employee left Dish Network ------------
* **January 2011 - October 2012** * Tier II Analyst / Developer II
* **Primary Operating Systems**: Solaris and RedHat Linux * Worked on a team of 8 to manage over 1,000 JVM application instances (WebMethods, Weblogic, and Tomcat) running on Solaris and Linux, writing Perl scripts to process log files, gather statistics, perform software upgrades, manage rolling restarts, etc. USDA ----
* _(Contract)_ **August 2010 - December 2010** * Windows Systems Administrator
* **Primary Operating Systems**: RedHat Enterprise Linux (version 5), Windows Server 2003 * Analyzed test environment needs to design scaled down test environments based upon network architecture diagrams of the proposed production environment * Constructed test clusters (one controller and two test agents each) for load testing with Visual Studio AMG Creative, Inc. ------------------
* **August 2009 - August 2010** * Web Developer / Systems Administrator
* Programmed websites on Joomla and Wordpress using PHP, JavaScript, HTML, and CSS. * Built and managed five servers (one virtual [Domain controller, file server, terminal server, backup server], another virtual [OpenVPN], one host [VMWare Server], one print server [Ubuntu 10.04], one backup server [Ubuntu 10.04]) CSU College of Business -----------------------
* **August 2007 - October 2009** * Web Developer / Systems Administrator
EDUCATION =========
Colorado State University -------------------------
* **2007-2010** * Bachelors in Business Administration
**Concentration**: Computer information Systems Courses included Windows and Linux administration, Networking, SQL database design and implementation, web application development (creating ASP.Net/C# and Cold Fusion applications that accessed a database and used sessions), client side application development (Visual Basic.Net), business math, marketing, finance, accounting, and business communication.
/* This resume was proudly generated with commonmark markdown, some clever css, and a Makefile */