Submodule from relative URL

[sp-ricard-valverde]

Git submodules allow relative URLs which are based on the parent repo URL.

Right now this is not supported, ie. adding a submodule configuration in .gitmodules with a relative URL will fail when cloning in the add or update Submodule.

A sample stack trace when calling submodule.update(init=True):

GitCommandError
Cmd('git') failed due to: exit code(128)
  cmdline: git clone -n --separate-git-dir=/opt/parent_project/.git/modules/submodule_project -v ../../my-company/submodule-project.git /opt/parent_project/submodule_project
  stderr: 'fatal: repository '../../my-company/submodule-project.git' does not exist
'
git/cmd.py in wait at line 291
git/util.py in finalize_process at line 341
git/repo/base.py in _clone at line 880
git/repo/base.py in clone_from at line 925
git/objects/submodule/base.py in _clone_repo at line 261
git/objects/submodule/base.py in update at line 524

GitPython==2.1.1
Still present in the current revision 8f76463

[PhilLab]

Similar but slightly different message in our system, when calling

REPO = Repo(ARGS.repositoryPath)
REPO.submodule_update(recursive=True)

File "C:\Program Files\Python36\lib\site-packages\git\repo\base.py", line 353, in submodule_update
    return RootModule(self).update(*args, **kwargs)
  File "C:\Program Files\Python36\lib\site-packages\git\objects\submodule\root.py", line 339, in update
    keep_going=keep_going)
  File "C:\Program Files\Python36\lib\site-packages\git\objects\submodule\root.py", line 201, in update
    smr.fetch(progress=progress)
  File "C:\Program Files\Python36\lib\site-packages\git\remote.py", line 789, in fetch
    res = self._get_fetch_info_from_stderr(proc, progress)
  File "C:\Program Files\Python36\lib\site-packages\git\remote.py", line 675, in _get_fetch_info_from_stderr
    proc.wait(stderr=stderr_text)
  File "C:\Program Files\Python36\lib\site-packages\git\cmd.py", line 415, in wait
    raise GitCommandError(self.args, status, errstr)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
  cmdline: git fetch --progress -v __new_origin__
  stderr: 'fatal: '../HelperModule.git' does not appear to be a git repository
fatal: Could not read from remote repository.

We are on a Gitlab instance where relative submodule URLs are a must have:

If you are using GitLab 8.12+ and your submodule is on the same GitLab server, you must update your .gitmodules file to use relative URLs

[sfuller]

I was able to workaround this issue by manually resolving the relative path in the .gitmodules file. If you care about the .gitmodules file retaining the relative path url, you could easily just set the original url back to the .gitmodules file after operating on the repo with gitpython.

for submodule in repo.submodules:

    # Hack around gitpython bug: https://github.com/gitpython-developers/GitPython/issues/730
    if submodule.url.startswith('..'):
        submodule_repo_name = submodule.url[3:]  # Strip off '../'
        repo_parent_url, _ = os.path.split(origin.url)
        actual_url = os.path.join(repo_parent_url, submodule_repo_name)
        with submodule.config_writer() as writer:
            writer.set('url', actual_url)

    print(f'Processing submodule {submodule.name}')
    submodule.update(init=True)

[Bruceforce]

Another workaround I'm currently using is to use git directly as described in https://gitpython.readthedocs.io/en/stable/tutorial.html#using-git-directly

for sm in repo.submodules:
      # Calling git directly for own submodules since using relative path is not working in gitpython
      # see https://github.com/gitpython-developers/GitPython/issues/730
      if sm.url[0:3] == '../':
          repo.git.submodule('init', sm.name)
          repo.git.submodule('update', sm.name)
      # For external submodules we can use the update function of gitpython
      else:
          sm.update()