O'Reilly 2016: "Continuous Delivery with Containers: The Trials and Tribulations" By Daniel Bryant

1.
Continuous Delivery forContainerized Applications: The Trials and Tribulations Daniel Bryant @danielbryantuk
2.
Setting the scene… •Continuous delivery is a large topic • Focusing on the process and tooling • No live coding today • Mini-book contains more details • “Building a CD pipeline” by Adrian and Kevin 20/12/2016 @danielbryantuk
3.
TL;DR – Containersand CD • Container image becomes the build pipeline ‘single binary’ • Adding metadata to containers images is vital • Must validate container constraints (NFRs) • Cultivate containerised ‘mechanical sympathy’ 20/12/2016 @danielbryantuk
4.
@danielbryantuk • Chief Scientistat OpenCredo, CTO at SpectoLabs • Agile, architecture, CI/CD, DevOps • Java, Go, JS, microservices, cloud, containers • Leading change through the application of technology and teams 20/12/2016 @danielbryantuk
5.
6.
Continuous Delivery • Producevaluable and robust software in short cycles • Optimising for feedback and learning • Not (necessarily) Continuous Deployment 20/12/2016 @danielbryantuk
7.
Creation of abuild pipeline is mandatory for continuous delivery 20/12/2016 @danielbryantuk
8.
9.
The Impact ofcontainers on CD 20/12/2016 @danielbryantuk
10.
Container technology (andCD) • OS-level virtualisation • cgroups, namespaces, rootfs • Package and execute software • Container image == ‘single binary’ 20/12/2016 @danielbryantuk
11.
12.
13.
14.
15.
Make your devenvironment like production • Develop locally or copy/code in container • Use base images from production • Must build/test containers locally • Perform (at least) happy path tests • All tests should be runnable locally 20/12/2016 @danielbryantuk
16.
Lesson learned: Dockerfilecontent is super important • OS choice • Configuration • Build artifacts • Exposing ports • Java • JDK vs JRE and Oracle vs OpenJDK • Golang • Statically compiled binary • Python • Virtualenv 20/12/2016 @danielbryantuk
17.
Please talk tothe sysadmin people: Their operational knowledge is invaluable 20/12/2016 @danielbryantuk
18.
Different prod andtest containers? • Create “test” version of container • Full OS (e.g. Ubuntu) • Test tools and data • Easy to see app/configuration drift • Use test sidecar containers instead • ONTEST proposal by Alexi Ledenev 20/12/2016 @danielbryantuk http://blog.terranillius.com/post/docker_testing/
19.
20.
Building images withJenkins • My report covers this • Build as usual… • Build Docker Image • Cloudbees Docker Build and Publish Plugin • Push image to registry 20/12/2016 @danielbryantuk
21.
Storing in animage registry (DockerHub) 20/12/2016 @danielbryantuk
22.
Lesson learned: Metadatais valuable • Application metadata • Version / GIT SHA • Build metadata • Build date • Image name • Vendor • Quality metadata • QA control • Security audited etc 20/12/2016 @danielbryantuk
23.
Metadata – Bewareof “latest” Docker Tag • Beware of the ‘latest’ Docker tag • “Latest” simply means • the last build/tag that ran without a specific tag/version specified • Ignore “latest” tag • Version your tags, every time • Danielbryantuk/test:2.4.1 20/12/2016 @danielbryantuk
24.
Metadata - AddingLabels at build time • Docker Labels • Add key/value data to image 20/12/2016 @danielbryantuk
25.
Metadata - AddingLabels at build time • Microscaling Systems’ Makefile • Labelling automated builds on DockerHub (h/t Ross Fairbanks) • Create file /hooks/build • label-schema.org • microbadger.com 20/12/2016 @danielbryantuk
26.
Metadata - AddingLabels at runtime 20/12/2016 @danielbryantuk $ docker run -d --label uk.co.danielbryant.lbname=frontdoor nginx • Can ’docker commit’, but creates new image • Not possible to update running container • Docker Proposal: Update labels #21721
27.
28.
29.
30.
31.
32.
33.
34.
35.
Mechanical sympathy: Dockerand Java • Watch for cgroup limits (and cgroup awareness) • getAvailableProcessors issue (bugs.openjdk.java.net/browse/JDK-8140793) • Default fork/join thread pool sizes (based from host CPU count) • Set container memory appropriately • JVM requirements = Heap size (Xmx) + Metaspace + JVM overhead • Account for native thread requirements e.g. thread stack size (Xss) • Entropy • Host entropy can soon be exhausted by crypto operations 20/12/2016 @danielbryantuk | @spoole167 35
36.
37.
38.
39.
Containerise an existing(monolithic) app? • For • We know the monolith well • Allows homogenization of the pipeline and deployment platform • Can be a demonstrable win for tech and the business • Against • Can be difficult (100+ line scripts) • Often not designed for operation within containers, nor cloud native • Putting lipstick on a pig? 20/12/2016 @danielbryantuk
40.
41.
Key lessons learned •Conduct an architectural review • Architecture for Developers, by Simon Brown • Architecture Interview, by Susan Fowler • Look for data ingress/egress • File system access • Support resource constraints/transience • Optimise for quick startup and shutdown • Evaluate approach to concurrency • Store configuration (secrets) remotely 20/12/2016 @danielbryantuk
42.
43.
Microservices… Containers and microservicesare complementary Testing and deployment change 20/12/2016 @danielbryantuk https://specto.io/blog/recipe-for-designing-building-testing-microservices.html
44.
45.
46.
47.
Microservice architectural impacton CD • Application decomposition • Bounded context • Change cadence • Risk • Performance • Scalability • Team location h/t Matthew Skelton, Adam Tornhill • Worth knowing about: • Consumer-based contracts • Service virtualisation • Synthetic transactions and semantic monitoring 20/12/2016 @danielbryantuk
48.
Using containers doesnot obviate the need for good architectural practices 20/12/2016 @danielbryantuk
49.
50.
51.
In summary • Continuousdelivery is vitally important in modern architectures/ops • Container images must be the (single) source of truth within pipeline • Mechanical sympathy is important (assert properties in the pipeline) • We’re now bundling more responsibility into our artifact (e.g. an OS) • Not all developers are operationally aware • The tooling is now becoming stable/mature • We need to re-apply old CD practices with new technologies/tooling 20/12/2016 @danielbryantuk
52.
53.
Thanks for listening •Any questions? • Feel free to contact me • @danielbryantuk • daniel.bryant@opencredo.com 20/12/2016 @danielbryantuk

O'Reilly 2016: "Continuous Delivery with Containers: The Trials and Tribulations" By Daniel Bryant

More Related Content

What's hot

Viewers also liked

Similar to O'Reilly 2016: "Continuous Delivery with Containers: The Trials and Tribulations" By Daniel Bryant

More from OpenCredo

Recently uploaded

O'Reilly 2016: "Continuous Delivery with Containers: The Trials and Tribulations" By Daniel Bryant