Filesystem Layouts

[rgajason]

Given the current discussions regarding CLI tools to help jump start projects, I wanted to share my team's experience with filesystem layouts in Starlite. This is just one opinion and not meant to be a criticism - people have different needs and situations.

The examples in the documentation largely demonstrate a project filesystem layout organized by framework function:

my_app
├── controllers
│   ├── component1.py
│   └── component2.py
└── models
    ├── component1.py
    └── component2.py

In this layout, imports would generally look like:

from my_app.models.component1 import Widget
from my_app.controllers.component1 import WidgetController

For my specific project/team, I/we found that this layout quickly left us with questions/issues that we couldn't solve in a way that made us happy.

Where would global dependencies/exceptions/responses/etc go?

If I have a directory called dependencies that holds my component dependencies I can't also have a top-level dependencies.py (without magic). Having dependencies/global.py or dependencies/my_app.py or other iterations of "shared", "common", etc felt wrong.

Where do things that aren't "native" to Starlite go?

We have an authentication component that uses CASBin. That library has a configuration file (model.conf - not to be confused with Starlite models). We also have a GitHub component that uses an auto-generated schema file using a utility from SGQLC (graphql_schema.py).

Where should these items live? They aren't models, controllers, or dependencies (in Starlite terms), and they aren't specific to Starlite or related to any other component in the app.

Alternate Layout

We landed on a filesystem layout organized by component. We think of these components as semi-pluggable "apps".

my_app
├── component1
│   ├── controllers.py
│   └── models.py
└── component2
    ├── controllers.py
    └── models.py

In this layout, imports would generally look like:

from my_app.component1.models import Widget
from my_app.component1.controllers import WidgetController

Some points we like about this layout:

• Component-specific stuff all lives in the same directory
• Decisions about where non-Starlite files should go or how they should be named are simpler
• Imports are sorted (isort) so that all imports from the same component are together
• Hierarchical layout where global dependencies, responses, exceptions, etc are at the top level and component-specific objects are grouped within their individual package

For our project, there will eventually be different development team owners of the different components. Organizing by component allows for us to put rules in GitHub's CODEOWNERS for approval purposes that reflect that ownership simply.

Here's a real-world example of this layout:

inventory
├── admin
│   ├── __init__.py
│   ├── controllers.py
│   ├── dependencies.py
│   ├── models.py
│   └── validators.py
├── auth
│   ├── __init__.py
│   ├── controllers.py
│   ├── dependencies.py
│   ├── model.conf
│   └── models.py
├── github
│   ├── __init__.py
│   ├── controllers.py
│   ├── dependencies.py
│   ├── graphql_schema.py
│   ├── models.py
│   └── validators.py
├── sandbox
│   ├── __init__.py
│   ├── controllers.py
│   ├── dependencies.py
│   └── models.py
├── templates
│   └── index.html.j2
├── __init__.py
├── config.py
├── dependencies.py
├── exceptions.py
├── main.py
└── responses.py

And an example of how imports shape up from our main.py:

import os

import uvicorn
from beanie import init_beanie
from motor.motor_asyncio import AsyncIOMotorClient
from starlite import (
    CORSConfig,
    LoggingConfig,
    Provide,
    Starlite,
    State,
    Template,
    TemplateConfig,
    get,
)
from starlite.contrib.jinja import JinjaTemplateEngine
from starlite.middleware import LoggingMiddlewareConfig
from starlite.status_codes import HTTP_500_INTERNAL_SERVER_ERROR

from inventory.admin.controllers import AdminController
from inventory.admin.models import Team, User
from inventory.config import AppSettings
from inventory.dependencies import get_settings, logging_exception_handler
from inventory.github.controllers import GitHubController, GitHubMetaController
from inventory.github.models import Organization, Repository
from inventory.sandbox.controllers import SandboxController
from inventory.sandbox.models import Child, Parent

[Goldziher]

I think dividing code by feature is usually the right way to go. This is the approach taken in different frameworks, e.g. django, nestjs or in such design patterns as MVVM. This correlates with the second option you've given.

In a nutshell:

repository_root
|-- pyproject.toml
|-- <config_files>
|-- src
|    |-- __init__.py
|    |-- main.py
|    |-- constants.py
|    |-- <shared_code>
|    |-- <feature_1_folder>
|    |-- <feature_2_folder>
|    |-- <feature_3_folder>
|-- tests
     |-- <feature_1_tests_folder>
     |-- <feature_2_tests_folder>
     |-- <feature_3_tests_folder>

You can also see a nestJS example i made a while back as an example for my colleagues: https://github.com/Goldziher/secret-note, it demonstrates a similar structure.