Git Fix Logs

Python scripts to assists with Git merging origin over source en-mass.

• when multiple remote repositories are missing license files, and/or commit history from a Fork gone wrong

• when one or two git push --force remote branch commands isn't enough

• for categorizing multiple repositories that cannot be automatically fixed and pushed; as well as why

---

• 🏗️ Requirements

• ⚡ Quick Start

• 🗒 Notes

• 🐚 Command Line Examples

• 📇 Attribution

• ⚖️ License

---

Requirements

The Git command line utility is required for doing version management stuff.

This script makes use of the following built-in Python libraries...

argparse
json
os
subprocess
sys

Pip install additional requirements via...

pip install --user -r requirements.txt

It is also a good idea to learn how to resolve merge conflicts

Optionally see Python Guide -- virtualenvs for information on running this project, and it's dependencies, within a Virtual Environment, TLD...

pip3 install --user pipenv

pipenv install -r requirements.txt

# pipenv run python3 fix_logs.py
# pipenv run python3 merge_failed.py

---

Quick Start

Make a place to clone this repository and download the source code...

mkdir -vp ~/git/hub/git-utilities


cd ~/git/hub/git-utilities


git clone [email protected]:git-utilities/fix_logs.git

Customize configuration file...

cd fix_logs


vim config.json

config.json

{
  "fixed": "./fixed.json",
  "failed": "./failed.json",
  "defaults": {
    "origin_branch": "master",
    "origin_remote": "origin",
    "source_branch": "master",
    "source_remote": "source",
    "fix_branch": "fix",
    "fix_commit": "Fixes logs",
    "keep_fix_branch": false,
    "no_push": false
  },
  "repos": [
    {
      "dir": "~/git/hub/llSourcell/Bitcoin_Trading_Bot",
      "source": "[email protected]:jaungiers/Multidimensional-LSTM-BitCoin-Time-Series.git"
    }
  ]
}

Example of adding to the list of repos...

{
  "fixed": "fixed.json",
  "failed": "failed.json",
  "defaults": {
    "origin_branch": "master",
    "origin_remote": "origin",
    "source_branch": "master",
    "source_remote": "source",
    "fix_branch": "fix",
    "fix_commit": "Fixes logs",
    "keep_fix_branch": false,
    "no_push": false
  },
  "repos": [
    {
      "dir": "~/git/hub/llSourcell/Bitcoin_Trading_Bot",
      "source": "[email protected]:jaungiers/Multidimensional-LSTM-BitCoin-Time-Series.git"
    },
    {
      "dir": "~/git/hub/llSourcell/How-to-Predict-Stock-Prices-Easily-Demo",
      "source": "[email protected]:jaungiers/LSTM-Neural-Network-for-Time-Series-Prediction.git"
    },
    {
      "dir": "~/git/hub/llSourcell/How_to_simulate_a_self_driving_car",
      "source": "[email protected]:naokishibuya/car-behavioral-cloning.git"
    }
  ]
}

---

Note, defaults may be overwritten for individual repos

    "repos": [
      {
        "dir": "",
        "source": "",
        "origin_branch": "master",
        "origin_remote": "hub",
        "source_branch": "tests",
        "source_remote": "source",
        "fix_branch": "fix-merge",
        "fix_commit": "Fixes logs",
        "keep_fix_branch": false,
        "no_push": true
      }
    ]

---

Run the fix_logs.py script...

python3 fix_logs.py --help


python3 fix_logs.py --config ./config.json

---

Notes

This project is not feature complete and is intended as a quick-fix/starting-point for those that didn't Fork repositories correctly. Pull Requests are welcomed for fixing bugs or adding features, or Open an Issue if assistance is needed with resolving bugs or adding features.

Success and failure logs are saved under ./fixed.json and ./failed.json by default. Though this may be modified by editing the config.json file to point to different paths.

Example fixed.json data...

{
  "fixed": [
    {
      "dir": "~/git/hub/account-name/Bitcoin_Trading_Bot",
      "source": "https://github.com/jaungiers/Multidimensional-LSTM-BitCoin-Time-Series.git",
      "origin_branch": "master",
      "origin_remote": "origin",
      "source_branch": "master",
      "source_remote": "source",
      "fix_branch": "fix",
      "fix_commit": "Fixes logs",
      "keep_fix_branch": false,
      "no_push": false,
      "code": 0,
      "err": "",
      "out": "Finished fixing /home/user-name/git/hub/account-name/Bitcoin_Trading_Bot"
    }
  ]
}

Example failed.json data...

{
  "failed": [
    {
      "dir": "/home/user-name/git/hub/account-name/repo-name",
      "source": "https://github.com/author/project.git",
      "origin_branch": "master",
      "origin_remote": "hub",
      "source_branch": "tests",
      "source_remote": "source",
      "fix_branch": "fix-merge",
      "fix_commit": "Fixes logs",
      "keep_fix_branch": false,
      "no_push": true,
      "message": "Cannot auto-merge <remote> <hash>",
      "code": "1",
      "err": "e.run['err']",
      "out": ""
    }
  ]
}

... It's a good idea to double check that fixed repositories genuinely have their logs corrected. And anything logged as failed should have Git logs corrected manually; check the Command Line Examples section of this document for hints on that.

---

Command Line Examples

Bash example for correcting a single git repository

cd ~/git/hub/llSourcell/Bitcoin_Trading_Bot

# Add `source` remote and fetch `master` branch history
git remote add source [email protected]:jaungiers/Multidimensional-LSTM-BitCoin-Time-Series.git
git fetch source master

# Last commit ref (git hash) of source/master
git checkout a8aaab3

# Use a temporary `fix` branch to _quarantine_ possible conflicts
git checkout -b fix-fork

# Merge last commit ref (git hash) of origin/master
git merge 91d9d98
# git merge -X theirs 91d9d98
# Note, using `-X theirs` strategy instead may reduce chances of conflict

#
## Handle any marge conflicts before proceeding
#

git commit -m 'Fixes logs'

# Checkout default branch and merge _fixes_
git checkout master
git merge fix-fork

# Delete _quarantine_ branch and force push to remote
git branch --delete fix-fork
git push --force origin master

Regardless of checked-out status one may use the following format...

# git log -1 --format='%h' <remote>/<branch>

... to obtain last hash for <remote>/<branch>

Note, above example's merge to last commit (hash 91d9d98) process will fast-forward any commits between detected divergence.

Merge conflicts may occur in which case vimdiff is an excellent command line utility for resolving differences manually; hint check for vimdiff cheat sheets with preferred web search service.

---

Attribution

• Git documentation -- merge strategies

• StackOverflow -- storing python dictionaries

• StackOverflow -- python try else

• StackOverflow -- how do you append to a file in python

• StackOverflow -- proper way to declare custom exceptions in modern python

• StackOverflow -- manually raising throwing an exception in python

• StackOverflow -- get exit code and stderr from subprocess call

• StackOverflow -- how to retrieve the hash for the current commit in git

• StackOverflow -- catch multiple exceptions in one line except block

• StackOverflow -- how to resolve merge conflicts in git

• StackOverflow -- How to get an absolute file path in Python

• StackOverflow -- Convert bytes to a string

• StackOverflow -- How can I find scripts directory with Python

• StackOverflow -- Bash -- Read a file line by line assigning the value to a variable

• StackOverflow -- Retroactively forking a project on GitHub

• Python Docs -- argparse

• Gist Cheat Sheet -- vimdiff

• Python Guide -- virtualenvs

---

License

Legal bits of Open Source software. Note the following license does not necessarily apply to any dependencies of this repository.

Fix Git Logs documentation
Copyright (C) 2020  S0AndS0

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published
by the Free Software Foundation; version 3 of the License.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU Affero General Public License for more details.

You should have received a copy of the GNU Affero General Public License
along with this program.  If not, see <https://www.gnu.org/licenses/>.

---

• ⬆️ Top of ReadMe File

---