Alembic integration and documentation

.github/workflows/test.yml

@@ -11,9 +11,9 @@ on:
   workflow_dispatch:
     inputs:
       debug_enabled:
-        description: 'Run the build with tmate debugging enabled (https://github.com/marketplace/actions/debugging-with-tmate)'
+        description: "Run the build with tmate debugging enabled (https://github.com/marketplace/actions/debugging-with-tmate)"
         required: false
-        default: 'false'
+        default: "false"
 
 jobs:
   test:
@@ -48,8 +48,10 @@ jobs:
         if: steps.cache.outputs.cache-hit != 'true'
         run: |
           python -m pip install --upgrade pip
+          python -m pip install types-setuptools
           python -m pip install "poetry"
           python -m poetry self add poetry-version-plugin
+
       - name: Configure poetry
         run: python -m poetry config virtualenvs.create false
       - name: Install Dependencies
@@ -78,7 +80,7 @@ jobs:
 
       - uses: actions/setup-python@v4
         with:
-          python-version: '3.8'
+          python-version: "3.8"
 
       - name: Get coverage files
         uses: actions/download-artifact@v3
@@ -100,7 +102,7 @@ jobs:
           path: htmlcov
 
   # https://github.com/marketplace/actions/alls-green#why
-  alls-green:  # This job does nothing and is only used for the branch protection
+  alls-green: # This job does nothing and is only used for the branch protection
     if: always()
     needs:
       - coverage-combine

.gitignore

@@ -12,3 +12,4 @@ coverage.xml
 site
 *.db
 .cache
+!docs_src/**/env.py

docs/advanced/index.md

@@ -5,6 +5,5 @@ The **Advanced User Guide** is gradually growing, you can already read about som
 At some point it will include:
 
 * How to use `async` and `await` with the async session.
-* How to run migrations.
 * How to combine **SQLModel** models with SQLAlchemy.
 * ...and more. 🤓

docs/advanced/migrations.md

@@ -0,0 +1,211 @@
+# Manage migrations
+
+SQLModel integrates [Alembic](https://alembic.sqlalchemy.org/en/latest/) to manage migrations and DB Schema.
+
+
+## **SQLModel** Code - Models and Migrations
+
+Now let's start with the SQLModel code.
+
+We will start with the **simplest version**, with just heroes (no teams yet).
+
+This is almost the same code as you start to know by heart:
+
+```Python
+{!./docs_src/tutorial/migrations/simple_hero_migration/models.py!}
+```
+
+Let's jump in your shell and init migrations:
+
+<div class="termy">
+
+```console
+$ sqlmodel migrations init
+Creating directory '/path/to/your/project/migrations' ...  done
+Creating directory '/path/to/your/project/migrations/versions' ...  done
+Generating /path/to/your/project/migrations/script.py.mako ...  done
+Generating /path/to/your/project/migrations/env.py ...  done
+Generating /path/to/your/project/migrations/README ...  done
+Generating /path/to/your/project/alembic.ini ...  done
+Adding '/path/to/your/project/migrations/__init__.py' ...  done
+Adding '/path/to/your/project/migrations/versions/__init__.py' ...  done
+Please edit configuration/connection/logging settings in '/path/to/your/project/alembic.ini' before proceeding.
+```
+</div>
+
+Few things happended under the hood.
+
+Let's review what happened: Below files just got created!
+
+```hl_lines="5-12"
+.
+├── project
+    ├── __init__.py
+    ├── models.py
+    ├── alembic.ini
+    └── migrations
+        ├── __init__.py
+        ├── env.py
+        ├── README
+        ├── script.py.mako
+        └── versions
+            └── __init__.py
+```
+
+Let's review them step by step.
+
+## Alembic configuration
+
+**`alembic.ini`** gives all the details of Alembic's configuration. You shouldn't \*have to\* touch that a lot, but for our setup, we'll need to change few things.
+
+We need to tell alembic how to connect to the database:
+
+```ini hl_lines="10"
+{!./docs_src/tutorial/migrations/simple_hero_migration/alembic001.ini[ln:1-5]!}
+
+#.... Lot's of configuration!
+
+
+{!./docs_src/tutorial/migrations/simple_hero_migration/alembic001.ini[ln:63]!} # 👈 Let's Change that!
+```
+Adapting our file, you will have:
+
+```ini hl_lines="10"
+{!./docs_src/tutorial/migrations/simple_hero_migration/alembic.ini[ln:1-5]!}
+
+#.... Lot's of configuration!
+
+
+{!./docs_src/tutorial/migrations/simple_hero_migration/alembic.ini[ln:63]!} # 👈 To that
+```
+
+For the full document, refer to [Alembic's official documentation](https://alembic.sqlalchemy.org/en/latest/tutorial.html#editing-the-ini-file)
+
+**`./migrations/env.py`** is another file we'll need to configure:
+It gives which Tables you want to migrate, let's open it and:
+
+1. Import our models
+2. Change `target_metadata` value
+
+```Python hl_lines="5 7"
+{!./docs_src/tutorial/migrations/simple_hero_migration/migrations/001.py[ln:1-5]!} 👈 Import your model
+# .....
+{!./docs_src/tutorial/migrations/simple_hero_migration/migrations/001.py[ln:19]!} 👈 Set you Metadata value
+```
+
+## Create an apply your first migration
+!!! success
+    👏🎉At this point, you are ready to track your DB Schema !
+
+Let's create you first migration
+
+<div class="termy">
+```console
+$ sqlmodel migrations init
+INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
+INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
+INFO  [alembic.autogenerate.compare] Detected added table 'hero'
+  Generating /path/to/your/project/migrations/versions/0610946706a0_.py ...  done
+
+```
+
+</div>
+
+Alembic did its magic and started to track your `Hero` model!
+It created a new file `0610946706a0_.py`
+
+
+```hl_lines="13"
+.
+└── project
+    ├── __init__.py
+    ├── models.py
+    ├── alembic.ini
+    └── migrations
+        ├── __init__.py
+        ├── env.py
+        ├── README
+        ├── script.py.mako
+        └── versions
+            ├── __init__.py
+            └── 0610946706a0_.py
+```
+
+Let's prepare for our migration, and see what will happen.
+
+<div class="termy">
+```
+$ sqlmodel migrations show
+Rev: 50624637e300 (head)
+Parent: <base>
+Path: /path/to/your/project/migrations/versions/0610946706a0_.py #👈 That's our file
+
+    empty message
+
+    Revision ID: 50624637e300
+    Revises:
+    Create Date: 2023-10-31 19:40:22.084162
+```
+</div>
+
+We are pretty sure about what will happen during migration, let's do it:
+
+<div class="termy">
+```
+$ sqlmodel migrations upgrade
+INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
+INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
+INFO  [alembic.runtime.migration] Running upgrade  -> 1e606859995a, migrating me iam famous
+```
+</div>
+
+
+Let's open our DB browser and check it out:
+
+<img class="shadow" src="/img/create-db-and-table-with-db-browser/image008.png">
+
+
+
+## Change you versions file name
+
+Why the heck `0610946706a0_.py`?!!!!
+
+The goal is to have a unique revision name to avoid collision.
+In order to have a cleaner file name, we can edit `alembic.ini` and uncomment
+
+```ini
+{!./docs_src/tutorial/migrations/simple_hero_migration/alembic.ini[ln:11]!} #👈 Uncoment this line
+```
+
+Let's remove `0610946706a0_.py` and start it over.
+
+<div class="termy">
+```console
+$ sqlmodel migrations revision
+INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
+INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
+INFO  [alembic.autogenerate.compare] Detected added table 'hero'
+  Generating /path/to/your/project/migrations/versions//2023_10_31_1940-50624637e300_.py ...  done
+```
+</div>
+
+Much better, not perfect but better.
+
+To get more details just by looking at you file name, you can also run
+
+
+<div class="termy">
+```console
+$ sqlmodel migrations revision "migrate me iam famous"
+INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
+INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
+INFO  [alembic.autogenerate.compare] Detected added table 'hero'
+  Generating /path/to/your/project/migrations/versions/2023_10_31_1946-1e606859995a_migrate_me_iam_famous.py
+  ...  done
+```
+</div>
+
+
+You can think of "migrate me iam famous" as a message you add to you migration.
+
+It helps you keep track of what they do, pretty much like in `git`

docs_src/tutorial/migrations/simple_hero_migration/alembic.ini

@@ -0,0 +1,116 @@
+# A generic, single database configuration.
+
+[alembic]
+# path to migration scripts
+script_location = migrations
+
+# template used to generate migration file names; The default value is %%(rev)s_%%(slug)s
+# Uncomment the line below if you want the files to be prepended with date and time
+# see https://alembic.sqlalchemy.org/en/latest/tutorial.html#editing-the-ini-file
+# for all available tokens
+file_template = %%(year)d_%%(month).2d_%%(day).2d_%%(hour).2d%%(minute).2d-%%(rev)s_%%(slug)s
+
+# sys.path path, will be prepended to sys.path if present.
+# defaults to the current working directory.
+prepend_sys_path = .
+
+# timezone to use when rendering the date within the migration file
+# as well as the filename.
+# If specified, requires the python-dateutil library that can be
+# installed by adding `alembic[tz]` to the pip requirements
+# string value is passed to dateutil.tz.gettz()
+# leave blank for localtime
+# timezone =
+
+# max length of characters to apply to the
+# "slug" field
+# truncate_slug_length = 40
+
+# set to 'true' to run the environment during
+# the 'revision' command, regardless of autogenerate
+# revision_environment = false
+
+# set to 'true' to allow .pyc and .pyo files without
+# a source .py file to be detected as revisions in the
+# versions/ directory
+# sourceless = false
+
+# version location specification; This defaults
+# to migrations/versions.  When using multiple version
+# directories, initial revisions must be specified with --version-path.
+# The path separator used here should be the separator specified by "version_path_separator" below.
+# version_locations = %(here)s/bar:%(here)s/bat:migrations/versions
+
+# version path separator; As mentioned above, this is the character used to split
+# version_locations. The default within new alembic.ini files is "os", which uses os.pathsep.
+# If this key is omitted entirely, it falls back to the legacy behavior of splitting on spaces and/or commas.
+# Valid values for version_path_separator are:
+#
+# version_path_separator = :
+# version_path_separator = ;
+# version_path_separator = space
+version_path_separator = os  # Use os.pathsep. Default configuration used for new projects.
+
+# set to 'true' to search source files recursively
+# in each "version_locations" directory
+# new in Alembic version 1.10
+# recursive_version_locations = false
+
+# the output encoding used when revision files
+# are written from script.py.mako
+# output_encoding = utf-8
+
+sqlalchemy.url = sqlite:///database.db
+
+
+[post_write_hooks]
+# post_write_hooks defines scripts or Python functions that are run
+# on newly generated revision scripts.  See the documentation for further
+# detail and examples
+
+# format using "black" - use the console_scripts runner, against the "black" entrypoint
+# hooks = black
+# black.type = console_scripts
+# black.entrypoint = black
+# black.options = -l 79 REVISION_SCRIPT_FILENAME
+
+# lint with attempts to fix using "ruff" - use the exec runner, execute a binary
+# hooks = ruff
+# ruff.type = exec
+# ruff.executable = %(here)s/.venv/bin/ruff
+# ruff.options = --fix REVISION_SCRIPT_FILENAME
+
+# Logging configuration
+[loggers]
+keys = root,sqlalchemy,alembic
+
+[handlers]
+keys = console
+
+[formatters]
+keys = generic
+
+[logger_root]
+level = WARN
+handlers = console
+qualname =
+
+[logger_sqlalchemy]
+level = WARN
+handlers =
+qualname = sqlalchemy.engine
+
+[logger_alembic]
+level = INFO
+handlers =
+qualname = alembic
+
+[handler_console]
+class = StreamHandler
+args = (sys.stderr,)
+level = NOTSET
+formatter = generic
+
+[formatter_generic]
+format = %(levelname)-5.5s [%(name)s] %(message)s
+datefmt = %H:%M:%S