Push large files to github: git-lfs
to answer to exceeding GitHub's file size limit of 100.00 MB
I am currently following deep learning specialization from Andrew Ng on coursera.
In the course 4 about CNNs, there are some pre-trained yolo models that we use to do object detection. And these models come as large .h5 files.
Because I run all programming assignments locally and keep everything (lectures + codes) on my local repo, when I pushed to github I got this error:
(base) explore@explore-ThinkPad-P53:~/git/guillaume/deeplearning_specialization$ git push
Enumerating objects: 247, done.
Counting objects: 100% (247/247), done.
Delta compression using up to 12 threads
Compressing objects: 100% (239/239), done.
Writing objects: 100% (242/242), 707.06 MiB | 4.74 MiB/s, done.
Total 242 (delta 6), reused 0 (delta 0)
remote: Resolving deltas: 100% (6/6), completed with 3 local objects.
remote: warning: File notebooks/C4W3/nb_images/pred_video.mp4 is 85.44 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: warning: File notebooks/C4W3/nb_images/road_video.mp4 is 81.71 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: error: GH001: Large files detected. You may want to try Git Large File Storage - https://git-lfs.github.com.
remote: error: Trace: 2d1944991c30279b831124b51e4aac57a17a860f2ef789b4e32801fb65282244
remote: error: See http://git.io/iEPt8g for more information.
remote: error: File notebooks/C4W2/ResNet50.h5 is 270.32 MB; this exceeds GitHub's file size limit of 100.00 MB
remote: error: File notebooks/C4W3/model_data/yolo.h5 is 194.69 MB; this exceeds GitHub's file size limit of 100.00 MB
To github.com:castorfou/deeplearning_specialization.git
! [remote rejected] master -> master (pre-receive hook declined)
As explained in https://github.com/git-lfs/git-lfs/wiki/Tutorial, there is (always) a way to do it properly.
First it is a matter of installing git-lfs:
sudo apt-get install git-lfs
Then to setup git lfs
git lfs install
And then to "migrate" big files to lfs:
git lfs migrate import --include="*.mp4"
git lfs migrate import --include="*.h5"
And now to git push
(base) explore@explore-ThinkPad-P53:~/git/guillaume/deeplearning_specialization$ git push
Uploading LFS objects: 100% (25/25), 954 MB | 37 MB/s, done.
Enumerating objects: 311, done.
Counting objects: 100% (311/311), done.
Delta compression using up to 12 threads
Compressing objects: 100% (273/273), done.
Writing objects: 100% (276/276), 60.49 MiB | 5.59 MiB/s, done.
Total 276 (delta 18), reused 0 (delta 0)
remote: Resolving deltas: 100% (18/18), completed with 16 local objects.
To github.com:castorfou/deeplearning_specialization.git
d0d2dc2..004fa09 master -> master