30 Mar 2019, 12:00

Deciding which Docker tag to use

In this post we’re going to talk about how versioning works in Docker container images. We’ll also explore some different strategies you will want to consider when using image version tags in your own Dockerfiles.

Docker container names are split into 2 parts: the name of the container image, and the version tag of the container image. For example, the container python:3 is a container hosted on Docker Hub at https://hub.docker.com/_/python. The name of the container is python, and 3 is the version tag. There are several other version tags of python also available, such as:

  • python:3.7
  • python:3.7.3
  • python:3-alpine
  • python:3.7-alpine
  • python:3.7.3-alpine
  • python:slim

Most of the popular languages use a similar tagging pattern in their official images on Docker Hub. Each of these tags represents different variations of an install of Python. Some use a different base operating system images (Alpine VS Debian VS “slimmed-down” Debian), and they also indicate the specific versions of whatever language is installed in the image.

Today the python:3 container image specifically uses Python version 3.8. Whenever 3.9 is released, that tag will get updated and the python:3 image will then contain version 3.9 instead. In a similar way, the python:3.7 image would be the latest available security patch of Python 3.7. Currently that happens to be 3.7.3, but tomorrow that could be 3.7.4 if a new version is released.

When writing your own applications, you will need to decide which version of the base image(s) you will use in the FROM statement(s) of your Dockerfile(s). The options include:

  • None (just use :latest, for example, just FROM python)
  • Specify a tagged version number (for example, python:3, or python:3.7, or python:3.7.3)
  • The exact sha256 digest of an image (for example, python@sha256:7e6c00cc553fdce06c1bcfcbf34c73a0f3623a8fc9ce88c8fb91671f2cbc0dab)

Let’s look a bit at each option, and discuss why you may want to use one tagging strategy over another.

19 Mar 2019, 10:17

Podcast: A big ol' cloud of compute

John and Justin made a guest appearance recently on the ‘Deloitte: On Cloud’ podcast with David Linthicum, talking about cloud native, DevOps, and Kubernetes from the business perspective.

What I want is just a big ol’ cloud of compute and I can just stuff my container in it and have it work. And we kind of have that now with Kubernetes. It’s not that there’s anything so magical about Kubernetes itself so much as the fact that it’s just universally adopted now.
—John

It was great for us to have a chance to talk about the organizational and business side of things—as David points out in the show, DevOps is always about the people. The ‘Cloud Native DevOps’ book is as much for IT directors, CTOs, and tech leaders as it is for engineers. If you’re not necessarily sold on the whole cloud and container thing, we explore the arguments for and against, and look at why, apart from sheer hype, Kubernetes is rapidly becoming the standard platform for modern apps. The book also explains how and why you need to transform your engineering organization along DevOps lines, and what that looks like in cloud native land.

Kubernetes is a no-brainer when you’re at the hyper-growth stage, and if your business is stable, it’s just the amount of time that you save by using a standard platform. There’s a migration cost, but I think there’s huge cost savings in the long run. It’s absolutely right to ask the questions, what is the ROI, when do we expect to see this thing pay off, and how do we know if we’re on the right track.
—Justin

Check out the podcast here:

Deloitte On Cloud Podcast: Cloud migration with containers and Kubernetes

If you don’t have time to listen, and we’re all busy people, you can also read the transcript of the show. Enjoy!

11 Mar 2019, 10:36

'Cloud Native DevOps with Kubernetes' is published!

The big day is here. It’s been a long time coming, but ‘Cloud Native DevOps with Kubernetes’ is finally published! Your pre-ordered copies should be with you very soon, if they’re not thumping on the doorstep already. If you didn’t pre-order, what were you thinking? Go ahead and buy the book right now:

Buy ‘Cloud Native DevOps with Kubernetes’ on Amazon.com

Buy ‘Cloud Native DevOps with Kubernetes’ on Amazon.co.uk

Cloud Native DevOps with Kubernetes: Building, Deploying, and Scaling Modern Applications in the Cloud

Some reader reviews:

The most encompassing, definitive, and practical text about the care and feeding of Kubernetes infrastructure. An absolute must-have.
—Jeremy Yates, SRE Team, The Home Depot

This book got me really excited. It’s a goldmine of information for anyone looking to use Kubernetes, and I feel like I’ve levelled up!
—Adam McPartlan (@mcparty), Senior Systems Engineer, NYnet

Cloud Native DevOps is an essential guide to operating today’s distributed systems. A super clear and informative read, covering all the details without compromising readability.
—Will Thames, Platform Engineer, Skedulo

Basically, this is the book we wish we’d had when we started trying to do stuff with Kubernetes. It explains the whole background to containers and Kubernetes, tells you what your options are for getting Kubernetes, what to do with it once you’ve got it, how to run it, how to deploy and scale your apps, how to secure your containers, how to monitor and gather metrics on your services, and much, much more! (See the table of contents)

Vital statistics:

  • Pages: 346
  • Chapters: 16
  • Words: 97,115
  • Writing time: 433hrs, 35m

03 Jan 2019, 09:36

2018 Kubecon

Some reactions from the recent Kubecon event in Seattle:

CNCF Projects are a good bet

With an ecosystem changing and splintering as fast as the cloud native world, the CNCF projects are a safe place to bet. They are stitching together a constellation of open source tools that you can use to build the foundations of your applications. All the pieces that you will need to glue together including the runtime, packaging, monitoring, tracing, etc. are there under the CNCF umbrella. Hopefully cloud providers will all rally behind these standards and focus collective efforts to maintain and improve them.

The community is diverging

It seems like there are 2 growing and diverging groups in the Kubernetes community: Those who have no idea where to start, and those who have already moved on to newer things. Lots of new folks are just getting started thinking about running applications in containers or starting to use a cloud provider, and others have been using Kubernetes for years and ready to jump into Knative or start tackling different software problems entirely. I imagine this usually happens in any fast-moving community.

I am grateful to all of those people who worked to make the event happen. Thank you to the presenters, cooks, cleaning staff, folks who worked at information desks, drivers, vendors, volunteers, and the organizers.

  • Justin

30 Dec 2018, 12:30

What's in the book?

We’re a few days away from going to press, so I thought you might like to see the up-to-date table of contents! If this whets your appetite for the book, why not pre-order your copy of ‘Cloud Native DevOps with Kubernetes’ now?

13 Dec 2018, 12:00

Database Migrations in Kubernetes

Applications with databases usually need to run migration tasks as part of their deployment processes. For example, in a Rails application this is done with the rake db:migrate command. Other frameworks have similar commands to manage the migrations. Typically, running migrations is one of the first steps in a deploy when upgrading the application to a new version. In a CI/CD pipeline for deploying an application running in Kubernetes there are a couple of options for how to handle migrations. In this post we’ll discuss two of them:

  • Running migrations from your CI/CD tool
  • Using a Kubernetes Job

CI/CD

This option involves adding a step to your CI/CD pipeline to run the migrations as part of a deploy. A common CI/CD workflow in Kubernetes looks something like this:

  1. Push code to source control
  2. Build a container with the new code
  3. Run the test suite
  4. Publish the container to a container registry
  5. Deploy the new container to a Kubernetes cluster with kubectl apply… or helm upgrade…

This last step is where the migration needs to happen. Jenkins, Drone, GitLab CI, or whatever CI/CD tool you use could migrate the database before setting a new version of your application to run. For example, you could add a step in the pipeline to run something like docker run <your-app-container>:<your-newly-built-container-tag> <your-db-migrate-command>....

The advantage to this method is that if the migration fails, the whole deployment fails and you can see right away that the deploy did not completely succeed. However, in order for this to be possible, your CI/CD tooling needs access to your application’s database, along with the application’s database credentials. This may not be possible if your CI/CD pipeline tools run in an isolated or separate environment from your application. For example, if you are using a SaaS CI/CD tool, network access to your application’s database may not be an option.

08 Dec 2018, 10:55

Securing Modern Applications in the Cloud

Kubernetes security is in the news at the moment, not necessarily in a good way. We thought it might be an appropriate time to share an interesting conversation we had recently with our friends Todd and Hannah at Threat Stack about Kubernetes, container security, and much more! We also had the fun of answering some questions from members of the audience.

Apart from hearing what we sound like, and in Justin’s case, look like, you’ll also hear a bit about how we came to write the ‘Cloud Native DevOps’ book, and exactly where we think security fits in to cloud native (spoiler: everywhere!)

Many thanks to DevOps.com for hosting the webcast, and to the magnificent Charlene O’Hanlon for being our moderator.

11 Sep 2018, 17:18

Order your copy now!

‘Cloud Native DevOps with Kubernetes’ is now available for pre-order! Despite what Amazon says, the first copies should be shipping in February, but make sure you get your order in now to beat the rush.

Pre-order ‘Cloud Native DevOps with Kubernetes’

Over the next couple of months we’ll be posting some teasers, previews, and exclusive extracts from the book here—so keep checking the blog!

12 Jun 2018, 08:56

It's official!

<img src="/static/img/titlepage.png" alt="" class="pure-img" >

It’s no longer just a dream. We’re delighted to announce that 'Cloud Native DevOps with Kubernetes' will be published by O’Reilly, hopefully early in 2019. Early release chapters will be available in Safari from around July 2018.

So, what’s it all about?

Cloud Native DevOps with Kubernetes

This is the second book you should buy after 'Kubernetes Up & Running'. 'Cloud Native DevOps' introduces the new world of cloud native applications and gives you a wealth of practical tips on how to run Kubernetes clusters, how to write and deploy Kubernetes applications, and how to apply DevOps best practices: infrastructure as code, continuous deployment, metrics, observability. We also cover managing cloud infrastructure with Terraform, migrating legacy applications to Kubernetes, building cloud native microservices in Go, and we provide complete, working example code for everything in the book, available in a separate GitHub repo. This book takes you from zero to Kubernetes in 210 densely-packed pages!

If you’re a cloud native novice, you’ll learn everything you need to know to get your application into the cloud. If you’ve been working with Kubernetes for a while, you’ll learn about the latest tools and features, community best practices, and battle-tested recipes from real production clusters. Even if you’re an experienced Kubernetes user, there’ll be a few things in this book that you didn’t know…​ we hope!

12 Mar 2018, 18:17

Don't panic!

Kubernetes is hard

If you’ve been hearing a lot about Kubernetes, and maybe read some things about it, or even tried to get started running something in Kubernetes, you may be feeling a little dispirited, confused, overwhelmed, puzzled, and possibly suffering the early symptoms of imposter syndrome. Well, don’t worry: it’s not just you.

Here’s the shocking truth that conference talks, promotional material, corporate press releases, and those other blogs won’t tell you:

Kubernetes is hard.

Of course, everyone who’s enthusiastic about Kubernetes wants to tell you how easy it is. “You can learn Kubernetes in a day!” Well, that would be quite a day. While Kubernetes is powerful and useful, it’s not necessarily so easy to get your head around, especially at first. It involves a lot of puzzling jargon and technical terms which don’t mean much to the newbie:

  • Pod
  • StatefulSet
  • PersistentVolumeClaim
  • Custom Resource Definition
  • Ingress
  • Deployment
  • Horizontal Pod Autoscaler

… and so on. If you feel depressed and angry at the amount of new things Kubernetes requires you to learn, then we can absolutely relate to that. We’ve been through some of the same emotions on our Kubernetes journey, which is by no means complete, but the point of this blog is to hammer a few signposts into the swamp to help others feeling similarly lost.