I've been using Vagrant to manage our development environment for ~2 years on several projects and like it. Works great when developing on OSX and using virtualbox/vmware as the virtualization providers. Development on Linux with vagrant-libvirt is not as well integrated, but with some manual steps it works.
As to handling incremental changes to the development environment, you could attempt to re-run the "vagrant provision" step and have it make your changes. This should be fine if you are using something like puppet/ansible/chef/salt to control the configuration of the VM. If you are using plain bash scripts you might be better served destroying the VM and re-creating. Creating a new VM is quick, ~1 minute (assuming your box has already been downloaded), then it's just the time for your provisioning scripts to run.
As to VM performance, the file system is often the bottleneck. You generally will want to share files between your host and your VM through some form of a "shared files" implementation. This could be NFS with libvirt, or the native shared folders implementation for virtualbox or vmware.....could also use rsync instead for any of the providers.
I'd recommend starting with vagrant + virtualbox, see how the performance is for you. Then consider tweaks if needed.
If you want to experiment with Docker you could try "fig" with docker, it is something like a vagrant for Docker. Of course vagrant supports Docker as well.
Happy to discuss this further if you run into any questions