Should I use Vagrant or Docker for creating an isolated environment? [closed]

Asked 2023-09-20 20:54:53 View 416,366

I use Ubuntu for development and deployment and have a need for creating an isolated environment.

I am considering either Vagrant or Docker for this purpose. What are the pros and cons, or how do these solutions compare?

  • Both are combinable now: docs.vagrantup.com/v2/provisioning/docker.html - anyone
  • Your question is lucky enough to get both of the writers' answers of the two services: Mitchell and Solomon Hykes - anyone
  • I would like to give a new summary - the question is mostly wrong. The right question is: Should I use Vagrant or docker-compose for creating an isolated environment? The answer is that Vagrant and docker-compose perform the same task of describing environments, and you should rather compare Docker to Virtualbox instead. The difference is that Vagrant can use any virtualization such as Docker, VMWare, Virtualbox on Windows, Linux or OSX, but docker-compose can just use Linux- based Docker images. - anyone
  • For me the the answer is "How important is speed for you in regular work activities". I find Vagrant to be slower than Docker. On docker, especially after an initial pull, the cache and layers approach of docker makes it the easiest and fastest for me as a developer to use it - anyone

Answers

Disclaimer: I wrote Vagrant! But because I wrote Vagrant, I spend most of my time living in the DevOps world which includes software like Docker. I work with a lot of companies using Vagrant and many use Docker, and I see how the two interplay.

Before I talk too much, a direct answer: in your specific scenario (yourself working alone, working on Linux, using Docker in production), you can stick with Docker alone and simplify things. In many other scenarios (I discuss further), it isn't so easy.

It isn't correct to directly compare Vagrant to Docker. In some scenarios, they do overlap, and in the vast majority, they don't. Actually, the more apt comparison would be Vagrant versus something like Boot2Docker (minimal OS that can run Docker). Vagrant is a level above Docker in terms of abstractions, so it isn't a fair comparison in most cases.

Vagrant launches things to run apps/services for the purpose of development. This can be on VirtualBox, VMware. It can be remote like AWS, OpenStack. Within those, if you use containers, Vagrant doesn't care, and embraces that: it can automatically install, pull down, build, and run Docker containers, for example. With Vagrant 1.6, Vagrant has docker-based development environments, and supports using Docker with the same workflow as Vagrant across Linux, Mac, and Windows. Vagrant doesn't try to replace Docker here, it embraces Docker practices.

Docker specifically runs Docker containers. If you're comparing directly to Vagrant: it is specifically a more specific (can only run Docker containers), less flexible (requires Linux or Linux host somewhere) solution. Of course if you're talking about production or CI, there is no comparison to Vagrant! Vagrant doesn't live in these environments, and so Docker should be used.

If your organization runs only Docker containers for all their projects and only has developers running on Linux, then okay, Docker could definitely work for you!

Otherwise, I don't see a benefit to attempting to use Docker alone, since you lose a lot of what Vagrant has to offer, which have real business/productivity benefits:

  • Vagrant can launch VirtualBox, VMware, AWS, OpenStack, etc. machines. It doesn't matter what you need, Vagrant can launch it. If you are using Docker, Vagrant can install Docker on any of these so you can use them for that purpose.

  • Vagrant is a single workflow for all your projects. Or to put another way, it is just one thing people have to learn to run a project whether it is in a Docker container or not. If, for example, in the future, a competitor arises to compete directly with Docker, Vagrant will be able to run that too.

  • Vagrant works on Windows (back to XP), Mac (back to 10.5), and Linux (back to kernel 2.6). In all three cases, the workflow is the same. If you use Docker, Vagrant can launch a machine (VM or remote) that can run Docker on all three of these systems.

  • Vagrant knows how to configure some advanced or non-trivial things like networking and syncing folders. For example: Vagrant knows how to attach a static IP to a machine or forward ports, and the configuration is the same no matter what system you use (VirtualBox, VMware, etc.) For synced folders, Vagrant provides multiple mechanisms to get your local files over to the remote machine (VirtualBox shared folders, NFS, rsync, Samba [plugin], etc.). If you're using Docker, even Docker with a VM without Vagrant, you would have to manually do this or they would have to reinvent Vagrant in this case.

  • Vagrant 1.6 has first-class support for docker-based development environments. This will not launch a virtual machine on Linux, and will automatically launch a virtual machine on Mac and Windows. The end result is that working with Docker is uniform across all platforms, while Vagrant still handles the tedious details of things such as networking, synced folders, etc.

To address specific counter arguments that I've heard in favor of using Docker instead of Vagrant:

  • "It is less moving parts" - Yes, it can be, if you use Docker exclusively for every project. Even then, it is sacrificing flexibility for Docker lock-in. If you ever decide to not use Docker for any project, past, present, or future, then you'll have more moving parts. If you had used Vagrant, you have that one moving part that supports the rest.

  • "It is faster!" - Once you have the host that can run Linux containers, Docker is definitely faster at running a container than any virtual machine would be to launch. But launching a virtual machine (or remote machine) is a one-time cost. Over the course of the day, most Vagrant users never actually destroy their VM. It is a strange optimization for development environments. In production, where Docker really shines, I understand the need to quickly spin up/down containers.

I hope now its clear to see that it is very difficult, and I believe not correct, to compare Docker to Vagrant. For dev environments, Vagrant is more abstract, more general. Docker (and the various ways you can make it behave like Vagrant) is a specific use case of Vagrant, ignoring everything else Vagrant has to offer.

In conclusion: in highly specific use cases, Docker is certainly a possible replacement for Vagrant. In most use cases, it is not. Vagrant doesn't hinder your usage of Docker; it actually does what it can to make that experience smoother. If you find this isn't true, I'm happy to take suggestions to improve things, since a goal of Vagrant is to work equally well with any system.

Hope this clears things up!

Answered   2023-09-20 20:54:54

  • @JaredMarkell I think maybe he's looking for a web-based service that lets him manage his Vagrant machines, such as Protobox. - anyone
  • @Mitchell I just wanted to say thank you for explaining this in such detail. Obviously it's impossible for you to be completely objective so I appreciate that you took time to explain the nuances and various situations where they could be used. I think a lot of the confusion around a variety of the tools today is that they overlap a lot, and a lot of people want a one-size-fits-all solution where somebody just tells them what to do and they can implement it. The beauty of your answer is that it answers the underlying question: how can I create an isolated environment? (irrespective of tools). - anyone
  • @JaredMarkell Docker has a REST API docs.docker.com/reference/api/docker_remote_api - anyone
  • @OğuzÇelikdemir Vagrant can do much more than that. Of course, if you prepare a specific virtual machine for every project, this will last. But during development I often end up adding more services / daemons / settings (eg. when I decide to use RabbitMQ for a project during development). Purely VM approach will require that you prepare a new image, with RabbitMQ installed and configured, and force developers to change their VM to this new one. For Vagrant - I'm adding appropriate lines in vagrant confguration and all developers can upgrade their VMs easily (using vagrant provision). - anyone
  • (You mean "disclosure", revealing something important, not "disclaimer", denying responsibility: english.stackexchange.com/q/115850) - anyone

I'm the author of Docker.

The short answer is that if you want to manage machines, you should use Vagrant. And if you want to build and run applications environments, you should use Docker.

Vagrant is a tool for managing virtual machines. Docker is a tool for building and deploying applications by packaging them into lightweight containers. A container can hold pretty much any software component along with its dependencies (executables, libraries, configuration files, etc.), and execute it in a guaranteed and repeatable runtime environment. This makes it very easy to build your app once and deploy it anywhere - on your laptop for testing, then on different servers for live deployment, etc.

It's a common misconception that you can only use Docker on Linux. That's incorrect; you can also install Docker on Mac, and Windows. When installed on Mac, Docker bundles a tiny Linux VM (25 MB on disk!) which acts as a wrapper for your container. Once installed this is completely transparent; you can use the Docker command-line in exactly the same way. This gives you the best of both worlds: you can test and develop your application using containers, which are very lightweight, easy to test and easy to move around (see for example https://hub.docker.com for sharing reusable containers with the Docker community), and you don't need to worry about the nitty-gritty details of managing virtual machines, which are just a means to an end anyway.

In theory it's possible to use Vagrant as an abstraction layer for Docker. I recommend against this for two reasons:

  • First, Vagrant is not a good abstraction for Docker. Vagrant was designed to manage virtual machines. Docker was designed to manage an application runtime. This means that Docker, by design, can interact with an application in richer ways, and has more information about the application runtime. The primitives in Docker are processes, log streams, environment variables, and network links between components. The primitives in Vagrant are machines, block devices, and ssh keys. Vagrant simply sits lower in the stack, and the only way it can interact with a container is by pretending it's just another kind of machine, that you can "boot" and "log into". So, sure, you can type "vagrant up" with a Docker plugin and something pretty will happen. Is it a substitute for the full breadth of what Docker can do? Try native Docker for a couple days and see for yourself :)

  • Second, the lock-in argument. "If you use Vagrant as an abstraction, you will not be locked into Docker!". From the point of view of Vagrant, which is designed to manage machines, this makes perfect sense: aren't containers just another kind of machine? Just like Amazon EC2 and VMware, we must be careful not to tie our provisioning tools to any particular vendor! This would create lock-in - better to abstract it all away with Vagrant. Except this misses the point of Docker entirely. Docker doesn't provision machines; it wraps your application in a lightweight portable runtime which can be dropped anywhere.

What runtime you choose for your application has nothing to do with how you provision your machines! For example it's pretty frequent to deploy applications to machines which are provisioned by someone else (for example an EC2 instance deployed by your system administrator, perhaps using Vagrant), or to bare metal machines which Vagrant can't provision at all. Conversely, you may use Vagrant to provision machines which have nothing to do with developing your application - for example a ready-to-use Windows IIS box or something. Or you may use Vagrant to provision machines for projects which don't use Docker - perhaps they use a combination of rubygems and rvm for dependency management and sandboxing for example.

In summary: Vagrant is for managing machines, and Docker is for building and running application environments.

Answered   2023-09-20 20:54:54

  • Just wanted to note that the Vagrant aspects of this answer are incorrect. Vagrant is not for managing machines, Vagrant is for managing development environments. The fact Vagrant spins up machines is mostly historic. The next version of Vagrant has first class support to spin up dev environment using Docker as a provider directly on the host or any VM (Mac, Win). It can also spin up raw LXC if thats what someone wants (again, on the host or VM). Vagrant is interested in doing what is best to create a portable development environment, whether that means creating a VM or not. - anyone
  • @Davide This covers it in more detail: vagrantup.com/blog/… - anyone
  • "It's a common misconception that you can only use Docker on Linux" While thats true, it is accurate to say that you can only use Linux on Docker. If I want to set up a test runner that exercises my application in a wide variety of envionment configs (different databases, php versions, caching backends etc), then I can easily do that with docker containers. But I cant see if my application will run properly in a windows IIS env, or on BSD or OSX. - anyone
  • Your first point is outdated since Vagrant has built-in provider support for docker: docs.vagrantup.com/v2/provisioning/docker.html - anyone
  • The post is outdated. Vagrant supports Docker as a provider now. And there are some videos demonstrating how you can use Vagrant and Docker in unison on their blog. - anyone

If your purpose is the isolation, I think Docker is what you want.

Vagrant is a virtual machine manager. It allows you to script the virtual machine configuration as well as the provisioning. However, it is still a virtual machine depending on VirtualBox (or others) with a huge overhead. It requires you to have a hard drive file that can be huge, it takes a lot of ram, and performance may be not very good.

Docker on the other hand uses kernel cgroup and namespacing via LXC. It means that you are using the same kernel as the host and the same file system. You can use Dockerfile with the docker build command in order to handle the provisioning and configuration of your container. You have an example at docs.docker.com on how to make your Dockerfile; it is very intuitive.

The only reason you could want to use Vagrant is if you need to do BSD, Windows or other non-Linux development on your Ubuntu box. Otherwise, go for Docker.

Answered   2023-09-20 20:54:54

  • Unfortunately, not yet. If you are in a 32 bit system, you will need a VM with a 64 bit guest system in order to run docker. However, with go1.1 the 32 bit support get better and it is possible that docker will support 32 bit arch soon - anyone
  • This is true for Mac and Windows, but since docker 0.7, any linux distro works fine. If you know of a non-working one, please let me know. Also, unless you have a Mac or Windows stack (which is unlikely but can happen), you do not want to run Docker anywhere but on linux. The docker client works fine on Mac, should work soon on BSD and the daemon will eventually support BSD, Solaris and Mac. - anyone
  • In case someone reads these comments, you should know that Docker hit ver1.0 just 12 days ago (blog.docker.com/2014/06/its-here-docker-1-0) and a whole lot of different platforms are stable & supported now (docs.docker.com/installation) - anyone
  • vagrant has LXC and docker provisioners. However - Vagrant and docker are fundamentally different things. Vagrant is purely for development environments, docker is rather for production and Linux only. - anyone
  • Docker now works on Windows 10 Pro and up and Windows Server 2016. I just upgraded from Windows 10 Home to Windows 10 Pro and installed the docker app. I can now run Linux docker images on Windows 10. It's brilliant! - anyone

I preface my reply by admitting I have no experience with Docker, other than as an avid observer of what looks to be a really neat solution that's gaining a lot of traction.

I do have a decent amount of experience with Vagrant and can highly recommend it. It's certainly a more heavyweight solution in terms of it being VM based instead of LXC based. However, I've found a decent laptop (8 GB RAM, i5/i7 CPU) has no trouble running a VM using Vagrant/VirtualBox alongside development tooling.

One of the really great things with Vagrant is the integration with Puppet/Chef/shell scripts for automating configuration. If you're using one of these options to configure your production environment, you can create a development environment which is as close to identical as you're going to get, and this is exactly what you want.

The other great thing with Vagrant is that you can version your Vagrantfile along with your application code. This means that everyone else on your team can share this file and you're guaranteed that everyone is working with the same environment configuration.

Interestingly, Vagrant and Docker may actually be complimentary. Vagrant can be extended to support different virtualization providers, and it may be possible that Docker is one such provider which gets support in the near future. See https://github.com/dotcloud/docker/issues/404 for recent discussion on the topic.

Answered   2023-09-20 20:54:54

  • Guys, I released an experimental vagrant provider for docker: github.com/fgrehm/docker-provider. - anyone
  • Docker is not virtualization, but a run the OS inside its own container, using same host kernel, its not a provider either like other VMs, so docker is already supported by Vagrant. - anyone
  • Docker is virtualization of the OS itself, implicitly reusing the underlying hardware. It is virtualization as it abstracts and isolates the filesystem, networking and processes running in a container. - anyone

They are very much complementary.

I have been using a combination of VirtualBox, Vagrant and Docker for all my projects for several months and have strongly felt the following benefits.

In Vagrant you can completely do away with any Chef solo provisioning and all you need your vagrant file to do is prepare a machine that runs a single small shell script that installs docker. This means that my Vagrantfiles for every project are almost identical and very simple.

Here is a typical Vagrantfile

# -*- mode: ruby -*-
# vi: set ft=ruby :
VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  config.vm.box = "mark2"
  config.vm.box_url = "http://cloud-images.ubuntu.com/vagrant/trusty/current/trusty-server-cloudimg-amd64-vagrant-disk1.box"
  [3000, 5000, 2345, 15672, 5672, 15674, 27017, 28017, 9200, 9300, 11211, 55674, 61614, 55672, 5671, 61613].each do |p|
    config.vm.network :forwarded_port, guest: p, host: p
  end
  config.vm.network :private_network, ip: "192.168.56.20"
  config.vm.synced_folder ".", "/vagrant", :type => "nfs"
  config.vm.provider :virtualbox do |vb|
    vb.customize ["modifyvm", :id, "--memory", "2048"]
    vb.customize ["modifyvm", :id, "--cpus", "2"]
  end
  # Bootstrap to Docker
  config.vm.provision :shell, path: "script/vagrant/bootstrap", :privileged => true
  # Build docker containers
  config.vm.provision :shell, path: "script/vagrant/docker_build", :privileged => true
  # Start containers
  # config.vm.provision :shell, path: "script/vagrant/docker_start", :privileged => true
end

The Bootstrap file that installs docker looks like this

#!/usr/bin/env bash
echo 'vagrant  ALL= (ALL:ALL) NOPASSWD: ALL' >> /etc/sudoers
apt-get update -y
apt-get install htop -y
apt-get install linux-image-extra-`uname -r` -y
apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 36A1D7869245C8950F966E92D8576A8BA88D21E9
echo deb http://get.docker.io/ubuntu docker main > /etc/apt/sources.list.d/docker.list
apt-get update -y
apt-get install lxc-docker -y
apt-get install curl -y

Now to get all the services I need running I have a docker_start script that looks somthing like this

#!/bin/bash
cd /vagrant
echo Starting required service containers
export HOST_NAME=192.168.56.20
# Start MongoDB
docker run --name=mongodb --detach=true --publish=27017:27017 --publish=28017:28017 dockerfile/mongodb
read -t5 -n1 -r -p "Waiting for mongodb to start..." key
# Start rabbitmq
docker run --name=rabbitmq --detach=true --publish=5671:5671 --publish=5672:5672 --publish=55672:55672 --publish=15672:15672 --publish=15674:15674 --publish=61613:61613 --env RABBITMQ_USER=guest --env RABBITMQ_PASS=guest rabbitmq
read -t5 -n1 -r -p "Waiting for rabbitmq to start..." key
# Start cache
docker run --name=memcached --detach=true --publish=11211:11211  ehazlett/memcached
read -t5 -n1 -r -p "Waiting for cache to start..." key
# Start elasticsearch
docker run --name=elasticsearch --detach=true --publish=9200:9200 --publish=9300:9300 dockerfile/elasticsearch
read -t5 -n1 -r -p "Waiting for elasticsearch to start..." key
echo "All services started"

In this example I am running MongoDB, Elastisearch, RabbitMQ and Memcached

A non-docker Chef solo configuration would be considerably more complicated.

A final big plus is gained when you are moving into production, translating the development environment over to an infrastructure of hosts that are all the same in that they just have enough config to run docker means very little work indeed.

If you interested I have a more detailed article on the development environment on my own web site at

Implementing A Vagrant / Docker Development Environment

Answered   2023-09-20 20:54:54

  • You did all of that docker_start orchestration but didn't bother linking the containers together. Are you just going with hard-coded port numbers because you're running it under Vagrant? - anyone
  • Hi WineSoaked, the example above does not show the container that actually uses all those services. If you look on the mentioned blog post there is another script script/vagrant/docker_web that fires up the development container for the project. This indeed does use --link on the docker run command and the Rails project uses the docker injected environment variables to connect up to the services. - anyone
  • I can see the potential merging both products. Vagrant as environment testing and docker for app wrapper. Merging both you can test a single app or unit-test on many escenarios. I think many of "testing platforms services" are using Vagrant+Docker at time. - anyone
  • "They are very much complimentary." — Both are free to use, indeed. - anyone
  • Hi @koppor I last used docker machine about three months ago and have not yet gone back to it. The problem I had was that it has a bug in the sharing of folders from my MAC host host to the VM running docker when using the VMWare driver. This meant that I could not edit code locally on the Mac and have the changes reflected in the docker container. I don't know if they have fixed it yet, when they do I will indeed switch to it. I have however switched all my container orchestration to docker compose since writing this answer - anyone

Vagrant-lxc is a plugin for Vagrant that let's you use LXC to provision Vagrant. It does not have all the features that the default vagrant VM (VirtualBox) has but it should allow you more flexibility than docker containers. There is a video in the link showing its capabilities that is worth watching.

Answered   2023-09-20 20:54:54

With Vagrant now you can have Docker as a provider. http://docs.vagrantup.com/v2/docker/. Docker provider can be used instead of VirtualBox or VMware.

Please note that you can also use Docker for provisioning with Vagrant. This is very different than using Docker as a provider. http://docs.vagrantup.com/v2/provisioning/docker.html

This means you can replace Chef or Puppet with Docker. You can use combinations like Docker as provider (VM) with Chef as provisioner. Or you can use VirtualBox as provider and Docker as provisioner.

Answered   2023-09-20 20:54:54

  • world just went crazy ;) we can run vagrant using docker provider to run docker containers inside vagrant - anyone
  • @zainengineer, does the Docker provider for Vagrant on Windows still use boot2docker or does it use some variant of Docker Toolbox? - anyone
  • @zainengineer Do you have some links to illustrative examples (not vagrant docs)? - anyone

Using both is an important part of application delivery testing. I am only beginning to get involved with Docker and thinking very hard about an application team that has terrible complexity in building and delivering its software. Think of a classic Phoenix Project / Continuous Delivery situation.

The thinking goes something like this:

  • Take a Java/Go application component and build it as a container (note, not sure if the app should be built in the container or built then installed to the container)
  • Deliver the container to a Vagrant VM.
  • Repeat this for all application components.
  • Iterate on the component(s) to code against.
  • Continuously test the delivery mechanism to the VM(s) managed by Vagrant
  • Sleep well knowing when it is time to deploy the container, that integration testing was occurring on a much more continuous basis than it was before Docker.

This seems to be the logical extension of Mitchell's statement that Vagrant is for development combined with Farley/Humbles thinking in Continuous Delivery. If I, as a developer, can shrink the feedback loop on integration testing and application delivery, higher quality and better work environments will follow.

The fact that as a developer I am constantly and consistently delivering containers to the VM and testing the application more holistically means that production releases will be further simplified.

So I see Vagrant evolving as a way of leveraging some of the awesome consequences Docker will have for app deployment.

Answered   2023-09-20 20:54:54

  • do you by any chance has a blog post about this? its been almost two years now, how its going? still using vagrant with docker or just docker and docker-fleat/machine? - anyone
  • The company I was working for was acquired and they took down all my content @Hoto. The short answer is I use docker-machine at home for my pet projects. At work I am <gulp>manager</gulp> and don't do much tech-ing. We do not have plans on using Docker so our tool is generally Vagrant. - anyone

Definitely Docker for the win!

As you may know Vagrant is for virtual machine management whereas Docker is for software containers management. If you are not aware of the difference, here is: A software container can share the same machine and kernel with other software containers. Using containers you save money because you don't waste resources on multiple operating systems (kernels), you can pack more software per server keeping a good degree of isolation.

Of course is a new discipline to care with its own pitfals and challenges.

Go for Docker Swarm if your requirements cross the single machine resources limit.

Answered   2023-09-20 20:54:54

There is a really informative article in the actual Oracle Java magazine about using Docker in combination with Vagrant (and Puppet):

Conclusion

Docker’s lightweight containers are faster compared with classic VMs and have become popular among developers and as part of CD and DevOps initiatives. If your purpose is isolation, Docker is an excellent choice. Vagrant is a VM manager that enables you to script configurations of individual VMs as well as do the provisioning. However, it is sill a VM dependent on VirtualBox (or another VM manager) with relatively large overhead. It requires you to have a hard drive idle that can be huge, it takes a lot of RAM, and performance can be suboptimal. Docker uses kernel cgroups and namespace isolation via LXC. This means that you are using the same kernel as the host and the same ile system. Vagrant is a level above Docker in terms of abstraction, so they are not really comparable. Configuration management tools such as Puppet are widely used for provisioning target environments. Reusing existing Puppet-based solutions is easy with Docker. You can also slice your solution, so the infrastructure is provisioned with Puppet; the middleware, the business application itself, or both are provisioned with Docker; and Docker is wrapped by Vagrant. With this range of tools, you can do what’s best for your scenario.

How to build, use and orchestrate Docker containers in DevOps http://www.javamagazine.mozaicreader.com/JulyAug2015#&pageSet=34&page=0

Answered   2023-09-20 20:54:54