First revision of Setup and Build document

[rschmied]

Documented the steps needed to bring up a working development environment from scratch. Uses a VM on top of a VMware hypervisor (for nested virtualization support).

[jainvipin]

@rschmied - thanks for the PR; this is useful!
What you describe would work to get the netplugin to compile and run unit tests. However for testing overlay/multi-host things, one would need to install etcd and consul as well across two such VMs that you describe. Those are very easy to install, perhaps it can be added right after this.

I am also in process of building a CentOS environment and will document the same for CentOS images.

[mapuri]

@rschmied thanks for the detailed write-up. It describes the build environment pretty well, that we just skim over in current README.md :( (here https://github.com/contiv/netplugin/blob/master/README.md#building-and-testing and here https://github.com/contiv/netplugin/blob/master/README.md#trying-it-out). It might be good to link your write-up in the README.md itself in the Building and Testing section.

A few minor knits

• all project docs (other than the main README) are currently in the docs dir, could you move this write-up there?
• change the PR's title to reflect it's purpose a bit more.

@jainvipin I think the intent of this doc is to just get user going with setting up their build environment. The dependency on etcd, consul etc are take care of by Vagrantfile and vagrant boxes themselves so they need not be documented here I think.

[rschmied]

Ok, I hope I did it right... :)

[mapuri]

An alternative that doesn't require creating symlinks is to checkout your fork in $GOPATH/src/github.com/contiv/netplugin directory as discussed here: #43 (comment)

[erikh]

There's an easier way:

$ export GOPATH=$HOME
$ go get -d github.com/contiv/netplugin
$ cd $HOME/src/github.com/contiv/netplugin
$ git remote add erikh git@github.com:erikh/netplugin

Then you work with remote erikh when pushing to your fork.

[erikh]

The advantage is that you're always tracking contiv/netplugin as master.

[mapuri]

LGTM. thanks! I realize it kind of closes the long standing #43 as well ☺️

[jainvipin]

@mapuri: Sure you can compile things on one host, but how do you develop something that needs to work across multiple hosts (clustered environment) you'll need etcd/consul. I am assuming someone wouldn't want to be happy with having compiled things and run unit-tests.

@jainvipin I think the intent of this doc is to just get user going with setting up their build environment. The dependency on etcd, consul etc are take care of by Vagrantfile and vagrant boxes themselves so they need not be documented here I think.

Regardless, this is a good start and we can build on top of this.

[mapuri]

I am assuming someone wouldn't want to be happy with having compiled things and run unit-tests.

Yes, I agree. But may be I missed something. If my understanding is correct, this document shall suffice for setting up the environment on single host to build netplugin, launch a (simulated) multi-host environment on that host and run all tests, without having to setup those clusters manually.

Are you looking for instructions to deploy etcd/consul in an actual bare-metal environment? I am not sure if we can provide instructions for that as those might be dependent on user's deployment environment like the network setup; cluster-discovery mechanisms etc.

[jainvipin]

@mapuri - I am not sure how to do multi-host testing without running etcd/consul based distributed kv store. Thus installing them is necessary, whether bare-metal or VMs. Or, may be I am missing something :-)

Yes, I agree. But may be I missed something. If my understanding is correct, this document shall suffice for setting up the environment on single host to build netplugin, launch a (simulated) multi-host environment on that host and run all tests, without having to setup those clusters manually.

[mapuri]

I am not disagreeing. You didn't answer my question :-)

Are you looking for instructions to deploy etcd/consul in an actual bare-metal environment?

CONTIV_NODES=2 make demo and CONTIV_NODES=2 make start-dockerdemo are two ways to setup a two node cluster using vagrant and docker-in-docker respectively, without user having to install or setup anything explicitly but it is a test/demo environment.

For a real deployment, I think we should rather have users follow instructions on etcd or consul website and follow the most up to date best practices of setting up a cluster suitable for their environment than having to keep those details here.

[jainvipin]

@mapuri - of course make demo creates multiple vagrant VMs from boxes
To answer your question - no, not just bare-metals :-)
I am looking for a common set of instructions for someone to bring up a set of machines (bare-metal or VMs) that can be used for development on a multi-clustered environment. Not just bare-metals, not real deployment, but for development.

For example, how can someone using a laptop/MAC create a development environment? Many who start working on netplugin face this problem. make demo wouldn't work IMO unless we start compiling netplugin on the laptop natively. If we can make make demo or equivalent to work with on a laptop, that would be wonderful. In short of that, including instructions to build a development clustered setup would suffice, which IMO should include installing and configuring etcd/consul.

[mapuri]

can be used for development on a multi-clustered environment.

hmm, I wonder what development in a multi-clustered environment really mean. I think you are still looking for a test/demo environment. And as I described earlier CONTIV_NODES=2 make demo and CONTIV_NODES=2 make start-dockerdemo shall suffice for that. Obviously, if user has a laptop with not enough horse power to run multiple VMs they shall be able to launch just the docker-in-docker test environment.

Anyways, I think we are agreeing that this document will help them get started with pulling the workspace, installing Go and other tools (vagrant, vritualbox etc) needed to build the code run unti tests and system tests (atleast dodcker-in-docker ones). I agree we can keep improving it or do separate write-ups as we find cases.

@rschmied
I think this is good to be merged barring one comment around the need to create symlinks for working on a fork. Would you be able to incorporate @erikh's note around alternative to creating symlink. I think it will help folks looking to contribute.

[erikh]

I’m going to take a stab at rebuilding our build system here in the next few days. I will be sure to update the documentation, but I don’t think the make targets will differ, just the plumbing.

-Erik

On Jul 16, 2015, at 5:43 PM, Vipin Jain notifications@github.com wrote:

@mapuri https://github.com/mapuri - of course make demo creates multiple vagrant VMs from boxes
To answer your question - no, not just bare-metals :-)
I am looking for a common set of instructions for someone to bring up a set of machines (bare-metal or VMs) that can be used for development on a multi-clustered environment. Not just bare-metals, not real deployment, but for development.

For example, how can someone using a laptop/MAC create a development environment? Many who start working on netplugin face this problem. make demo wouldn't work IMO unless we start compiling netplugin on the laptop natively. If we can make make demo or equivalent to work with on a laptop, that would be wonderful. In short of that, including instructions to build a development clustered setup would suffice, which IMO should include installing and configuring etcd/consul.

—
Reply to this email directly or view it on GitHub #119 (comment).

[rschmied]

@mapuri: added @erikh 's method into the doc. should be good to go, I hope.

[rschmied]

k... one more try :)

[mapuri]

LGTM

@rschmied can you also please squash all your commits to a single commit to get this ready for merge? thanks.

[erikh]

Yeah that's perfect, sorry to make this so complicated! LGTM after squash

[rschmied]

Dabbling into and still learning the intricacies of Git. I think that I should memorize this part. If I got it right, I would rebase my source tree since I have had the local changes in my fork already? Anyway, more dabbling required ;)

[erikh]

Here are some (hopefully useful) instructions:

$ git checkout master
$ git pull --rebase
$ git checkout <your branch>
$ git rebase -i master

At this point your editor will open and you'll be asked to edit the text to provide rebase instructions. What you want to do is replace the pick with s on every line but the first one. After saving and exiting, some operations will occur and you will see another editor to edit your commit message.

Then,

$ git push -f <your remote> <your branch>

And that should trigger CI and we can merge!

[rschmied]

OK, guys... I'm a bit at a loss here. I guess I messed it up by adding things to my fork in the master branch instead of adding it to a new / specific branch. I followed Erik's instructions and while doing so I ended up in a 'detached HEAD' state with the recommendation to create a new branch. Which I did. So I now have a new branch in my fork called 'doc-changes' with all my changes / commits nicely squashed. It now tells me that I could create a new PR with those changes. Which probably would have been OK if I did not open THIS PR in the first place. Wondering if there's any way out of this e.g. if I could revise or reset something.

Please advise. Thanks

[erikh]

Hey, I've tried to contact you on cisco jabber so I can walk you through this, but you were offline. Alternatively, I am happy to finish up the git stuff for you.

[rschmied]

Hi, Erik... Please go ahead and do the needful. I will be out of the office next week anyway. But I would be eager to learn how to actually tackle this. So, if you can give me an idea what I would have to do: I'd appreciate that.

[erikh]

Sure. To repair your branch, it'll take this. However, Please read the rest of this before executing the statements below.

$ git rebase --abort
$ git checkout master
$ git reset --hard <your remote>/master
$ git rebase -i <our remote>/master

So the important thing is that you ensure you are using the right remotes.

$ git remote

Will show you the names. If you do not have a remote set up for us, do this:

git remote add netplugin https://github.com/contiv/netplugin
git fetch netplugin

Now netplugin will be the name for "our" remote, and origin is probably your remote.

You can verify which remote points at what by typing:

$ git remote show <remote>

It will tell you what repository it refers to and a lot of other data.

HTH! Please let me know if you are still lost.

[shaleman]

Closing this since we have pretty good developer documentation at https://github.com/contiv/netplugin/blob/master/docs/DevEnv.md

@rschmied Feel free to reopen or file an issue if you see some info missing.

-sukhesh