Folder within folder issue with SCM

Unreal Courses Talk
#1

@sampattuzzi just thought I’d point out a couple of fixes for the folder within a folder issue. You can create the project before cloning your empty repo and merely initialize and set an origin repo and push to it.

e.g.

cd myNewProject
git init
git add .
git commit -am 'Initial Import'
git remote add origin git@mygitrepo.com:myorg/myrepo.git
git push -u origin main

or if you have already got a commit in your repo (with perhaps a default license or readme), just clone the repo next to your new repo to a tmp directory:

git clone git@mygitrepo.com:myorg/myrepo.git myTmpDir

and copy its .git directory into your project:

cp -av myTmpDir/.git myNewProj/
cd myNewProj
git status

the above git status will note the missing readme and license files, copy them as well if you wish:

cp ../myTmpDir/README.md ./
cp ../myTmpDir/LICENSE ./

Now git status should show all your new project files, this would be a great time to add a .gitignore file. I’ve notice that the default projects also include a .ignore file containing:

/.vscode
/Content
/Intermediate

which I’m uncertain whether they should be ignored in git as well?

PS - there are other ways to get around the origin already having commits, git push -f, deleting branches etc…

#2

Content is needed. If you ignore this, all blueprints will be lost and indeed all assets. I usually add everything in the content folder but some git hosts may not support repos that large.

Usually I use the format **/path/ when adding the ignore.

Lastly, I find it easier to create the repo first with a default git ignore from whatever host you are using then clone it. Then, create the project. Init works too however.

I hope this makes sense.

#3

I think the problem with cloning first is that unreal will create a directory within your repo leading to paths that look like:

/f/unrealprojects/myrepo/myrepo

Which is where I usually let unreal create the one myrepo directory and then introduce the .git directory inside it to prevent this behavior. If there is a way to get unreal to avoid making the directory inside of the repo and just use the repo directory itself as the project directory I’d definitely like to know about it.

#4

If you are going to do that then rather than copy the git folder, something you should never touch, use git init to initialise the folder once you set up the project. Then, you can copy in a git ignore file and perform a commit locally as normal. On gitlab, github or whatever provider, create the blank u initialised repo. Finally, you map the git folder to this repo and push.

I will reiterate, don’t ever copy the git folder. You should never mess with that folder manually.

Git has all the necessary commands to do exactly what you need without messing with the git internal folder.

The commands you want look something like…


git remote add origin https://github.com/somethinghere/repo.git
git push -u origin main
#5

I will add the -u origin as in that scenario I imagine the upstream would be unset. I’ve been messing around in the .git directory for nearly two decades, and I have not had any issues whatsoever copying .git directory over as I described above. git status will tell you all you need to know about what changes git perceives.

FWIW - I also edit .git/config directly often as I see fit to change remotes and branches. Yes there are commands to do the same, but I am simply faster and more comfortable with vim and I don’t have to remember cryptic syntax (other than the cryptic vim commands I already use on a daily basis).

#6

And the problem still remains that if you git init and add your remote that already has files in it (like your gitignore) you will get the ‘fatal: refusing to merge unrelated histories’ error, leading back to either a -f force push, or some gymnastics like I did where I pulled .git directory in (along with the .gitignore and any other initial files). So far as I know it, there are no git commands that will simply do that.

#7

That’s fine if you are experienced and you feel comfortable doing it but For those who aren’t, they should not.

Been using scm in one form or another for about 3 decades and git for the last 10 years and I wouldn’t do this nor would I recommend beginners to do it this way. The simple reason when it goes wrong, they will expect us TAs to fix it. Also, linux users (I assume linux, possibly mac due to the vim reference) are usually more adept at hacking around these sorts of things and most users here will be windows, probably 95% or more.

#8

I definitely prefer linux in many ways, but first and foremost that all configs are plain text. This is true for .git/config as well, go look at it, you should not be afraid of looking under the hood. I’d say the number one problem with hiring people fresh out of college is that they might be able to program but they know next to nothing about actual systems. I’ve said it many times, you cannot be a decent programmer without being a decent sysadmin, and vice versa.

On another branch in this topic, I think you will find that unreal projects are a bit too large to be hosted at any of the free providers without dipping into a paid plan pretty quick, personally I’m running a gitea helm chart on trueNAS.

#9

Azure devops is fine. No limit in size. I’ve have repos that exceed 30gb without issue.

#10

You must have a paid account:

Enumerating objects: 297, done.
Counting objects: 100% (297/297), done.
Delta compression using up to 20 threads
Compressing objects: 100% (283/283), done.
remote: TF402462: This push was rejected because its size is greater than the 5120 MB limit for pushes in this repository. Learn more at https://aka.ms/gitlimit
send-pack: unexpected disconnect while reading sideband packet
fatal: sha1 file '<stdout>' write error: Broken pipe
error: remote unpack failed: error TF402462: This push was rejected because its size is greater than the 5120 MB limit for pushes in this repository. Learn more at https://aka.ms/gitlimit
fatal: the remote end hung up unexpectedly

^this was a test upload of the MLdeformer sample project which is over 30GB in size onto a azure devops repo.

#11

Anybody who realizes the value of actually owning and holding your own data should check out gitea, furthermore, I can recommend the implementation from Truecharts on TrueNAS.

If you represent an organization that needs much more, take a look into running your own gitlab, though it is much more resource intensive than gitea.

If you are looking for much less try out gitolite (possibly the lightest git server implementation):

https://gitolite.com/

#12

I really don’t. I signed up years ago however so perhaps they’ve changed the limits

#13

In any case, I think it is unwise to expect 30GB repos to be handed out for free for very long. Gitlab was giving out free 10GB repos until recently when they decided there was way too much data being stored on their servers for free.

I did run a kubernetes based gitlab setup for a bit, but it was five machines and way too much redundancy for my small team needs.

I do have a gitolite instance on another VM that was very appealing in its lightweight nature. Very easy to maintain and configure and almost zero resources dedicated to it and yet able to handle very large repos with ease.

Right now though I have a middle ground that is gitea/trueNAS on a $200 minipc with two 14TB drives on a ZFS mirror and a replicated offsite 14TB at another geographic location for disaster recovery. Dirt cheap at under $1k, geographically redundant, and decently spacious 14TB. I might add that I have this gitea instance authing off of an ldap instance (also running in the TrueNAS) which allows me to add my users to multiple services at once (not just gitea). Something that might interest anyone setting up multiple services with LDAP is Authelia!

Privacy & Terms