Designated Initializer & C++
Build Native Android Programs with CMake and android-cmake

Repost: Speed Up Git (5x to 50x)

tubo posted @ 2014年10月15日 14:46 in 未分类 , 1370 阅读

 

| Comments

Note: Results may vary, depending on distance from your Git servers. In my completely unscientific benchmarks using time, after the following steps, git pull went from ~5s, using GitHub, to ~0.1s, using EC2 on AWS Singapore.

Why?

1
2
3
4
$ time git pull
Already up-to-date.

real    0m5.075s

5 seconds just to tell you that your Git repository is already up-to-date? Unacceptable.

Enable SSH Connection Sharing and Persistence

In Singapore, the round-trip time to github.com is ~250ms. Establishing an SSH connection every time you perform a Git operation costs many round-trips, but you can keep them around and reuse them with the following lines in ~/.ssh/config:

~/.ssh/config
1
2
3
ControlMaster auto
ControlPath /tmp/%r@%h:%p
ControlPersist yes

ControlMaster auto enables the sharing of multiple SSH sessions over a single network connection, and auto-creating a master connection if it does not already exist.

ControlPath /tmp/%r@%h:%p specifies the path to the control socket used for connection sharing. %r will be substituted by the remote login username, %h by the target host name and %p by the port.

ControlPersist yes keeps the master connection open in the background indefinitely.

With this, git pull goes down to ~1s, a 5x improvement. Of course, this speeds up your other repeated SSH connections too!

Setup Git Mirror with Automatic Mirroring to GitHub

For sub-second Git remote operations, you’re going to need a server with low latency. (Imagine, when physically working together, you could git pull the very moment someone says “Pushed!”.)

However, you can still have all the niceties GitHub provides, by automatically pushing to GitHub from your server. There are several alternatives for keeping GitHub updated, like using cron jobs or multiple remotes, but I think using a post-receive hook is the most elegant solution.

On your server, set up a GitHub deploy key:

1
2
$ ssh-keygen -t rsa -C "your_email@example.com"
$ cat .ssh/id_rsa.pub

Go to your target repository’s Settings –> Deploy Keys –> Add deploy key and paste the public key in and submit. Then you can mirror your repository:

1
$ git clone --mirror git@github.com:ahbeng/example.git

To automatically to mirror to GitHub after you’ve pushed, set up a Git hook at hooks/post-receive:

hooks/post-receive
1
2
#!/bin/bash
nohup git push --mirror &>/dev/null &

Your Git client won’t disconnect till the script has completed, so a simple git push --mirror would defeat the purpose of setting this up, since you’d be adding on GitHub’s latency to your pushes again. So, we background the process and keep it running after logging out using nohup, and redirect all I/O streams to /dev/null to prevent SSH from hanging on logout.

Make the hook executable:

1
$ chmod +x hooks/post-receive

Now you can use your new Git mirror locally. Assuming you have added the server’s private key to ssh-agent:

1
$ ssh-add ec2.pem

You could clone it:

1
$ git clone user@hostname:example

Or change your existing repository’s remote:

1
$ git remote set-url origin user@hostname:example

Results

Taken alone, using the Git mirror brings git pull down to ~1.2s. When combined with SSH connection sharing:

1
2
3
4
$ time git pull
Already up-to-date.

real    0m0.111s

0.1s Git remote operations. Awesome.

 

 

celeb networth 说:
2021年8月31日 02:51

If you are interested in your favorite actors' net worth and detailed information, please take a look at celeb networth post database.

kelloggs family rewa 说:
2022年8月25日 05:43

Kellogg's Family Rewards 2021 :These free Kellogg's Family Rewards codes will help you fill your mailbox with free stuff that you use your points to redeem. This includes free gift cards, toys A LITTLE MORE OF WHAT YOU LOVE. kelloggs family rewards codes NEW Rice Krispies Treats® Homestyle Original and Chocolate bars are 50% bigger than our 22g Original bar 2021.Kellogg's Family Rewards 2021 :These free Kellogg's Family Rewards codes will help you fill your mailbox with free stuff that you use your points to redeem. This includes free gift cards

CGBSE 12th Model Pap 说:
2022年9月05日 01:06

CG Board Model Paper 2023 Class 12 Pdf Download with Answers for English Medium, Hindi Medium, Urdu Medium & Students for Small Answers, Long Answer, Very Long Answer Questions, and Essay Type Questions to Term1 & Term2 Exams at official website. CGBSE 12th Model Paper New Exam Scheme or Question Pattern for Sammittive Assignment Exams (SA1 & SA2): Very Long Answer (VLA), Long Answer (LA), Small Answer (SA), Very Small Answer (VSA), Single Answer, Multiple Choice and etc.

Emma 说:
2023年1月22日 02:59

This article has provided an excellent solution to the problem of slow Git speeds. After applying the steps suggested, I experienced a dramatic improvement in my Git speeds, going vet CBD product from 5 seconds to 0.1 seconds while using GitHub and EC2 on AWS Singapore. This is an amazing result and is sure to be a great help to many Git users.

meidir 说:
2023年2月08日 03:01

Wonderful article! thank you for sharing your knowledge with us SaaS Contracts

Jimmy 说:
2024年4月14日 10:02

Enabling SSH connection sharing and persistence in ~/.ssh/config optimizes Git operations, especially in locations like Singapore with high round-trip times to github.com. By keeping connections active and reusing them, you minimize the overhead of establishing new connections, enhancing efficiency in Git workflows.. Cornelius North Carolina Real Estate

 


登录 *


loading captcha image...
(输入验证码)
or Ctrl+Enter