I’m leading a team of 6 developers who are working in different geographies across different working hours, and the best investment I’ve made on behalf of the company so far, is getting us on GitHub. Just to give you some background information, our platform, BitcoinFundi, is built on Ruby on Rails but it’s more than just a Rails & MySQL application. It has different components and custom daemons so to get all of this to work together, our only option was to build our own Ubuntu servers on Amazon’s AWS.
Our Server Environments
We have two servers: A development/testing server we call Dionysius and the production server we call Mercury. All of these are connected to one git repository on GitHub and we call this repository Origin. Origin contains BitcoinFundi’s source code.
Dionysius is a near identical clone of Mercury and , besides them both running two different branches of Origin, one big difference between the two is that Dionysius has a PhantomJS installation that uses the Capybara gem to run automated tests on all the source code that is upload by developers before it’s tested by a human being prior to deployment on Mercury.
How we use branches on Servers & Dev Machines
We were able to achieve this by having two branches on Origin – Master & Production. The active branch on Dionysius is Master branch because that’s the branch developers push their changes to.
Developers are free to create as many branches on their local computers – we’re very easy going about how developers do stuff locally. I, for instance, mostly just work on one branch called Tawanda and when I write code I want to test on Dionysius. I commit my code, switch to Master, merge Master with Tawanda and then push Master to Origin. There’s another developer on our team who prefers to create a different branch for every different feature he’s working on. Developers can do what they want on their local machines – as long as they are pushing comits to Master, we don’t care.
How we automated deployments
When a push is made to the Master branch on Origin, it triggers a webhook that gets Dionysius to automatically pull the changes that were made to the source code and then reload the web server. After that, PhantomJS lauches Capybara to test the code for bugs we can find programmatically before an actual human being steps in. If there are any errors, we get an email notification and attend to that before we push another commit. When we’re finished with our Slack integration, we’ll be able to get these notifications in Slack. That will be nice!
If the tests succeed, the developer logs into Dionysius to check that everything is indeed working the way it’s supposed to work on Mercury. At that point, the developer creates a pull request, gets someone else to review the changes and merge the pull request with Production branch to effect those changes on Mercury.
If you would like to implement this in your organisation too then Github has an excellent tutorial on how to deliver automated deployments to your server.
When a new developer joins our organisation
First they install Git on their local machine
Downloading & installing git is pretty straitforward
Next they add ssh keys to GitHub
We have a private repository on Github so only developers that are granted access will be able to push and pull from Origin. But first, the developers will need to upload their public ssh keys on GitHub. GitHub has an easy-to-follow tutorial on how to add ssh keys to GitHub.
Then they set up defaults
Assuming the dev’s name is Tinashe Dube and their email address is tin@bitfinance.co.zw, they run the following commands:
Then they clone the Git repository
Here they:
- Clone origin and put the source code in a folder called bitcoinfundi
- Move to the bitcoinfundi folder
They run the following commands:
# git clone git@github.com:BitFinance/BitconFundi.git bitcoinfundi
# cd bitcoinfundi
Then they make sure Git knows who they are
It’s important for us to know who pushed what commits to Origin so we have everyone run the following commands to tell Git who they are:
# git config --global user.name "Tinashe Dube"
# git config --global user.email tin@bitfinance.co.zw
Set default branch for pushing and pulling
I heard it said once that the Scots spell whisky without the ‘e’ (whisky not whiskey) because they believe that adding the ‘e’ wastes drinking time. I also heard that if you see a bottle of whiskey written “Scotch Whisky” then it’s a fake and not really from Scotland because the Scots just call it “Scotch” because they believe that adding the word whisky wastes drinking time. I don’t know how far true this is but at BitFinance we have the same mentality. We hate to waste drinking time by specifing the branch we’re pushing or pulling to whenever we push or pull commits. So we want a situation where:
- If no branch is specified we want to push to the current branch we are on – in this case it is the Master branch
- If no branch is specified when we run git pull, pull new commits on the Master branch
To do this, we run the following commands:
# git config --global push.default matching
# git branch --set-upstream-to=origin/master master
BitFinance’s 3 rules for commits
We’ll obviously continue to add/remove/modify these rules and grow and learn more but right now we’re living by these rules:
- We use GitHub’s issues to keep track of what tasks the developers should be working on. Developers should only be working on one of the open issues.
- Developers should make as many commits as possible but should only push when they have something they want to test on Dionysius.
- The commit message should contain what they worked on and the issue number preceded by a hash. For example:
# git commit -m "configuring routes to fix issue #4"
or
# git commit -m "generation error controllers and views for issue #4"
We do this because when anyone opens the commit from Github they will see a clickable link to the associated issue. This makes it easier to track and close issues.
BitFinance’s 4 rules for creating issues
- An issue should be created for anything the development team should be working on.
- On creation, an issue should have as much of the following information as is possible:
- A short descriptive name and a longer detailed description of what needs to be done.
- The issue should be assigned to somebody
- A label to show the type of issue it is. The most common types of issues will be
- Bug
- Enhancement
- Documentation
- New Feature
- Cleanup/Refactor
- Incomplete Feature awaiting Completion Issue type is an important determiner of urgency & importance so in most cases a Bug will be more urgent than a New Feature
- The milestone before which this new issue should be completed
- In the description, the links should have links to references if any are available.
- Comments section is used by team members to debate/discuss issues
BTW, we’re hiring so if you’re a Rails developer and working with Bitcoin and these technologies are something that excites you, then I’d encourage you to consider joining our team. We have both part-time and full-time positions so you don’t necessarily have to leave your job to start working with us.
I have been writing code for the last thirteen years and I still remember how hard it was to collaborate with other people on the same project before Git came along. Especially if you were all editing the same source files. Humanity has really come a long way and technologies like Git have made the world a better place.
The author, Tawanda Kembo, is co-founder of BitFinance, a startup that is making it super easy for people in Africa to start using Bitcoin.
Share
Home
13 comments
Tawanda, l have no clue what you are talking about and l cannt get my head around this github thing bt all l know its loved developers.
Im managing a project being funded by EU and one of my developers is the original founder of hdfs which most people know as hadoop.
I build a test cluster of servers and within days he wanted a Github account and for “security”(read: no scooby what is was) reasons l said NO. I proposed to build him any internal one and did the easiest way:
https://www.turnkeylinux.org/blog/github (you can find anything here)
They used it and after a while l agreed to open them a online version. Up to now l just do not get. But work is being, code is being shared and making progress.
I’ll try to explain what Git & GitHub is and hopefully this explanation will make sense to you.
Let’s say you’re building one website with only one file and lets call this file index.html. You create the first version of index.html which has 100 lines of html in it and then you upload it to a server. Everything works fine and everybody is happy. Every week, your boss asks you to add information to this new file and it keeps getting bigger and bigger until one day it is 1000 lines of code. One day your boss hires an accountant and asks you to add the account to the website. The names of staff members is somewhere in the middle of the website so you add one line of code to make index.html 1001lines of code. The next day, we realise that the accountant lied about something in her CV and she is fired. You now have to remove that line of code you added so you download index.html and look through hundreds of lines of code looking for that one line and then you delete it. Wouldn’t it have been easier if you could just hit undo?
Now your boss decides to use the budget he had created for the accountant salary to hire an additional developer and you think it’s a weight off your shoulders but when you upload the index.html file you have removed the accountant’s name, the new web developer uploads the same file with his name on it. Because you had both downloaded the same index.html file and you both made changes to the same file but you edited different lines, your file is going to be overwritten by the file that is uploaded by the new developer.
Git has many different features and advantages but i this example I have only just mentioned two problems it solves (We need a whole blog post explaining Git). The first problem is rolling back changes. imagine if your boss were to go on leave for 6 months and then suddenly come back and say that I want the website where it was 6 months ago – where would you start? Git tracks all the changes you made to your website so you can roll back changes to any point in history you want. This is very cool and especially useful when something suddenly breaks and you don’t understand why.
The second problem is that of resolving conflicts – what happens when we both edit different lines in the the same file? or the same lines or the same lines in the same file? How do we avoid overwriting each other’s changes? Git solves both this problem and software that solves this problem is called a Version Control System. Git will keeps track of the specific changes that were made to a file so it will know that you removed the accountants name from index.html and that the new developer added his name to index.html and Git knows where exactly in the file those changes were made so when you push changes to the web server, Git only applies those specific changes and doesn’t really move the whole file. Both these feature are very useful if you have a 2+ of developers working on the same source code. At BitFinance we edit files without worrying about who else is editing those files because we know that Git will take care of that. We are also not afraid to screw up because we can just roll back to the last point in history that was working fine – this gives us freedom to experiment and try stuff and let Git take care of the last thing that worked.
And GitHub is just the world’s biggest Git-as-service website. I like it because it has some very cool features. My favourite feature is their built in issue tracking system we use to keep track of outstanding jobs and who should be working on it. My next favourite feature is their built in wiki – we use that for documentation.
I hope this explanation is helpful
Hey!! Thanks a lot. I have a progress review meeting coming up and lm an expert already. Vachandiona…
That is a very simple, noble educational infor.
Thanks again
We live & learn
Dude !,
the location ascribed to this piece of very interesting news is Zimbabwe.
Considering what you are working on, this is amazing news.
I had to go back and reread the background information,
as Zimbabwe is almost always alleged to be
one of the few places on the planet
where anything of this sort would seem impossible if not completely illegal.
That you are so far out there is a refreshing bit of news,
keep up the good works.
Thanks for the encouragement.
You may also want to know that a Zimbabwean lawyer, Simba Machiridza did some research that covered two of the issues you have mentioned: is this possible in Zim? Is it illegal? and he published a whitepaper on that. In this whitepaper he looks at how Zim’s population is mobile money savvy, how they are used to dealing with different currencies in the multicurrency system they use and a few other problems that can be solved using Bitcoin and he concludes that Zimbabwe is the country that has the best use case for Bitcoin in Africa.
He also analyses the legal framework and whether the current legal framework and the concludes that the laws currently in place in Zimbabwe would permit the application of Bitcoin, should the authorities adopt an open interpretation of their provisions.
You can read that whitepaper here: http://www.slideshare.net/simbarashemachiridza/bitcoin-white-paper?qid=b0f1021a-2bbc-4336-a38d-b1dc016f501d&v=default&b=&from_search=1
Thanks for the encouragement.
You may also want to know that a Zimbabwean lawyer, Simba Machiridza did some research that covered two of the issues you have mentioned: is this possible in Zim? Is it illegal? and he published a whitepaper on that. In this whitepaper he looks at how Zim’s population is mobile money savvy, how they are used to dealing with different currencies in the multicurrency system they use and a few other problems that can be solved using Bitcoin and he concludes that Zimbabwe is the country that has the best use case for Bitcoin in Africa.
He also analyses the legal framework and whether the current legal framework and the concludes that the laws currently in place in Zimbabwe would permit the application of Bitcoin, should the authorities adopt an open interpretation of their provisions.
You can read that whitepaper here: http://www.slideshare.net/simbarashemachiridza/bitcoin-white-paper?qid=b0f1021a-2bbc-4336-a38d-b1dc016f501d&v=default&b=&from_search=1
Great stuff!!
keep up This good work.
hi Tawanda
I have a keen interest in cryptocurrencies and the uses of non-fiat money. I would love to join your organisation as a volunteer since I am currently studying and might not have enough in terms of time or expertise.
I use github myself but mainly for adding to the malware analysis repository theZoo and helping out on computer forensics based projects.
I hope you succeed in your endeavours and go far. Just don’t pull any funny business like what happened at Mt.Gox
What happened at Mt. Gox was sad but kuipa kwezvimwe ndiko kunaka kwezvimwe because the whole industry has learnt from the mistakes Mt. Gox made.
What happened to Gox won’t happen to us because we take security very seriously. In fact, we’re probably the most secure Bitcoin Exchange on the planet. I wrote a whole blog post talking about our aproach to Bitcoin security. You can read it here: http://bitfinance.co.zw/why-we-chose-to-secure-our-hot-wallet-with-bitgo/
About joining our team, I’d love to have that conversation. I think it will be an exciting learning opportunity for you because you’ll get to work with some very smart people. Are you free to have a Skype call (on that topic) with me next Wednesday or next Sunday? If you are, you can book a meeting with me here: http://meetme.so/tawandakembo
FYI, we have a whatsapp group called Bitcoin Zimbabwe. If you would like to connect with other people involved in different Bitcoin-related projects in Zimbabwe, then email me your mobile number so that I add you to this group
I think it is a good model for a start. Over time you would have challenges of availability during deployment. What happens when the new code has passed Unit, Integration tests but you realise that something in regression is broken. I assume you have a QA team that does regression and you alsoo use headless phantomjs to run system tests.
If you do that, this may be a long process, may take 30 minutes. What happens to transactions that run during this moment? My suggestion is to have 4 environments Dev, Staging,Integration and Production. Instead of autodeploy to production, you can do that whenever a feature is added to develop. Follow the GitFlow System. Each developer works on a feature when it is done, they merge into develop and trigger the hooks on integration environment.
Instead of requiring the same repo across environments, you can require the same dev system across developers. You can achieve this by virtualization, (look into Vagrant ). So all your devs will be running their code on eg CentOS even if developing on Windows/Ubuntu. Look into Chef for automated deployment or Puppet. I know more chef than puppet and it is ruby based.
Finally, source control is the way to go, I am glad you shared this
Thank you for sharing this Freeman. This is very helpful feedback to heart as we’re learning and growing.
I’m glad I wrote this article because now we’re getting useful feedback.