Why Containers?

  • Steve Jones - SSC Editor

    SSC Guru

    Points: 719107

    Comments posted to this topic are about the item Why Containers?

  • ktflash

    Ten Centuries

    Points: 1127

    Possibly dumb question, but i don't understand why and i am curious.

    What is the reasoning behind creating these posts before it is actually available?

    • Are they some sort of reminder for yourself?
    • Are they some kind annoucement? If so are they only made for certain topics?

     

    I wanna be the very best
    Like no one ever was

  • David.Poole

    SSC Guru

    Points: 75348

    We have template Python projects so that the start of any Python development has a big head start

    • Virtual environment build (venv)
    • Test framework install with sample test
    • Library install
    • Linter (flake8)
    • build and test from a simple command line
    • Container build
    • Container push to the container repository

    Because we are using niche technology I have had to write my own equivalent of Redgate SQLDoc, SQL Source Control and a deployment pipeline.  In the production environment the containers to execute that functionality spin up when required, do their job, terminate and evaporate.  Because everything is self-contained there is no danger of the libraries from one app clashing with the libraries of another app.

    My colleagues have experimented with minimising the container size so we use Linux Alpine as a base container.  The trade off is that an app with many dependencies may take some development effort to get working due to the minimal install of the initial container but once it's done it's done.

    Docker images can be very fast to spin up and very fast to execute what they do.  They reward those who write tight, efficient code and protect systems from those who don't.

    We do run databases in containers for our local development though not in production.  It used to be the case that Docker containers should be regarded as stateless but this is no longer the case.

    The container approach also means that I can have more of the software stack running on my workstation.  This means that not only do I have great freedom to play around I can do so without impacting other people.  I can learn with bravery.  It also means that I am not reliant on any form of network connection to do my work.

    What comes after containers?  Unikernels.  These are apps with their own OS but only the OS needed to do what the apps are intended to do.  If your app doesn't need a display, printer, keyboard or mouse then it won't have the OS mechanisms or drivers for those things.  The pay off is that a unikernal app can be tiny as in single digit Kb.  This means they are blazingly fast to boot up.  They carry out their tasks and shut down.  Not only do they have less to hack they don't exist long enough to be hacked.

    If you want to learn about Docker and Kubernetes I can recommend Nigel Poulton's courses on Pluralsight and his books.

  • MVDBA (Mike Vessey)

    SSC-Insane

    Points: 21757

    we're writing up a plan for using kubernetes, docker and lambda functions in AWS (not a live product yet) - I'm looking forward to it

    - ktflash , these are 100% available … someone correct me if i'm wrong but sql 2019 and kubernetes play nicely together

     

    MVDBA

  • call.copse

    SSCoach

    Points: 17152

    I'll look those courses up David - if I ever get the 2 hours necessary - thanks.

  • Jeff Moden

    SSC Guru

    Points: 996449

    David.Poole wrote:

    We have template Python projects so that the start of any Python development has a big head start

     

      <li style="list-style-type: none;">

    • Virtual environment build (venv)

     

      <li style="list-style-type: none;">

    • Test framework install with sample test

     

      <li style="list-style-type: none;">

    • Library install

     

      <li style="list-style-type: none;">

    • Linter (flake8)

     

      <li style="list-style-type: none;">

    • build and test from a simple command line

     

      <li style="list-style-type: none;">

    • Container build

     

      <li style="list-style-type: none;">

    • Container push to the container repository

    You mean that Python installations are that complex?

    Also, @Steve... with all this wonderful talk, do you think you good folks could finally fix this site so that old code and quoted things, like the one above, don't have a bazillion extra line feeds in them?  It's ridiculous how bad some of the old code windows are.

    --Jeff Moden


    RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
    First step towards the paradigm shift of writing Set Based code:
    ________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.
    "Change is inevitable... change for the better is not".

    Helpful Links:
    How to post code problems
    How to Post Performance Problems
    Create a Tally Function (fnTally)
    Forum FAQ

  • This was removed by the editor as SPAM

  • Steve Jones - SSC Editor

    SSC Guru

    Points: 719107

    David.Poole wrote:

    If you want to learn about Docker and Kubernetes I can recommend Nigel Poulton's courses on Pluralsight and his books.

     

    Second vote for Mr Poulton. He does a great job and I've enjoyed his courses.

    More of the stack is good, but there are still resource issues, especially with data. However, dropping a set of containers on servers for a dev is certainly easier than installing everything.

  • MVDBA (Mike Vessey)

    SSC-Insane

    Points: 21757

    if you are looking at docker and kubernetes and you are thinking about using amazon aws then start looking at AWS fargate

    MVDBA

  • Steve Jones - SSC Editor

    SSC Guru

    Points: 719107

    If you are looking at kubernetes, look at AWS, Azure, or GCP. Not worth running yourself in most cases.

     

  • David.Poole

    SSC Guru

    Points: 75348

    No Jeff, we do it that way to ensure enforced  consistencyfor all data engineers, abstracting the docker process, making sure that any engineer can run the test/build on their laptop.

Viewing 11 posts - 1 through 11 (of 11 total)

You must be logged in to reply to this topic. Login to reply