Shipping Deis with Docker

Matthew Fisher

CEO, Fishworks

matthewf@opdemand.com

@bacongobbler

About Me

  • CEO of Fishworks
  • Self-employed contractor
  • bacongobbler on github, IRC, twitter
  • Open contributor to Deis since v0.1.1

About Deis

  • github.com/deis/deis
  • First Multi-host Docker PaaS
  • Heroku-inspired workflow with Docker goodies
  • 100% free (as in beer) and open source
  • Python and Go
    • Looking to migrate to Go soon™

Deis Today

  • 2500 stars, 150 watchers, 300 forks
  • 5 full-time devs
  • 16 unique contributors in the last month
  • > 100 Deis deployments "in the wild" (some in prod, too!)

How did we get here?

Platform Approach

  • Should be lightweight
  • Make the world pluggable
  • Use tools which people are comfortable with
  • Hacking should be a breeze
  • Focus on working code, optimize later

Devops Approach

  • Developers own the application
  • Operations own the platform

Rush to Prototype

        
            $ deis register http://deis.local
            $ deis keys:add ~/.ssh/id_rsa.pub
            $ deis create myapp
            $ git push deis master
            $ deis config:set DATABASE_URL=postgres://
            $ deis scale web=8
        
    

What did we need?

  • API Server
  • Command-line Interface
  • Docker
  • Git Server
  • Heroku Buildpacks
  • Scheduler
  • Service Discovery
  • Router

Where does Docker fit in?

App Deployment Workflow

  • Receive an application's source code
  • Compile app to a heroku slug
  • Inject slug into Docker container
  • Inject runtime envvars in last mile stretch
  • Deploy app to a remote Docker cluster
  • Publish app to service discovery
  • Expose app through the router

Environment Variable Injection

  • In-house registries accept JSON metadata on PUT requests
  • Therefore, with a little black magic...
  • HOWTO inject envvars

    Source code courtesy of deis/deis
            
                import requests, json
                latest_image = json.loads(requests.get('http://registry.example.com/v1/repositories/sample/tags/latest').text)
                data = json.loads(requests.get('http://registry.example.com/v1/images/{}/json'.format(latest_image)).text)
                data['config']['Env'].append('HELLO=world')
                requests.put('http://registry.example.com/v1/images/{}/json'.format(latest_image), json.dumps(data))
                _commit('busybox', data, _empty_tar_archive(), 'latest')
            
        
            
                $ docker run registry.example.com/busybox:latest echo $HELLO
                world
            
        

    Is this Optimal?

    • No, not by any stretch
    • Focus on working code first, optimize later

    More Features!

    • deis logs
    • deis run
    • deis releases
    • deis sharing

    Deis v0.5.0

    Docker gets Stable

    Time to Dockerize Deis!

    Deis Components

    • deis/builder
    • deis/cache
    • deis/controller
    • deis/database
    • deis/discovery
    • deis/logger
    • deis/registry
    • deis/router

    Deis Scalability Issues

    • Max of 20 servers or 5 deploys/minute
    • Node convergence is slow
    • Hard to recover from failed convergence
    • Parallel convergence = disaster
    • No disaster recovery
    • No cluster re-balancing
    • Etc...

    TL;DR Chef != job scheduler

    Chef == Config Management

    Technical Debt

    • Shared directories -> images
    • Databag scheduler -> Fleet
    • Chef -> /dev/null

    We need Fleet, Etcd and Docker. Hmm...

    Deis v0.11.0

    • Deployed at companies of all sizes
    • Active user and developer communities
    • Growing rapidly

    Major features as of late

    • Deis pull
    • Build apps from Dockerfile
    • Logger re-written in Go
    • App memory/CPU limits
    • Tag-based scheduling
    • Deisctl

    How about a Demo?

    Future Plans

    Mesosphere Integration

    • Identical workflow, enterprise-grade scheduler
    • Mesos is in full production at Twitter, Hubspot, Airbnb
    • Working closely with Mesosphere team

    CoreUpdate

  • Based on the Google Omaha protocol
  • Can be used to update Fleet units...
  • ...Perfect for in-place upgrades!
  • Things we learned

    • Documentation and tests first, code later
    • Pay attention to the ecosystem
    • Invest heavily in automated testing

    Interested in Contributing?

    • Come talk to me (I won't bite!)
    • Ping me on IRC, email, twitter
    • Pull up an easy-fix issue
    • We're also hiring...

    Resources

    Thanks!