TODO.org

Esganalysis

Fixes

API

Ready on postgresql health check

check if model id exists in text/sentences

NO Run tox command before startup (for parsers)

• commands

NO add midelware with timings

[#A] add repr to database

[#A] change psycopg2 to asyncpg

pytest async

change status code to request.status

Setup tox

Move send/get requests from class to utils

Проверить как работает парсер МФО для banki.ru

Когда выгружал сделал text.id = text_sentence.id

Проверить как парсит остальное irecommend

parser sentences background task

async alembic

bulk insert/update

add FastAPI annotations 0.95.x

• release notes

Tests

faker + factoryboy

refactor to commit to db instead of api

add different types of type_bank_id

logic tests

Parser

update parser state last

browser only for one bank

rename parser utils to common

Refactor vk parser

Use subclass for requests

Tests

tesing parser (link)

add pytest command for parsers link

integration test in different folder link

vcr.py (link)

add readme for subprojects

speed up tests

make .env file with common variables

check mfo parser licence

Doc

детализация актуальности

• не понятно какие сервисы
• Что понимается под этичностью (дать определение)
• Существуют сервисы, которые оценивают компанию по судебным делам (?), но не дают оценку по отзывам
• Уточнить цель работы

+

Plans

Must

Sqlalchemy naming convention

alembic migrations

SQLalchymy 2.0

Nginx reverse proxy (link)

• on method

different workers for read and write (cqrs)

compress request data with nginx?

nginx reverse proxy (load balancer(?)) maybe there

https://youtu.be/RfN0B7L4SuM?list=PLEwK9wdS5g0riA4Q_fqcjkv0zYf6HgRGJ

Отдельный воркер для обработки отызвов в предложения

Change docker compose variables for env_file

• env variables in grafana

Citation for libraries

Maybe

Fast api dependencies (link)

compress large sentences with gzip header (maybe with nginx)

customize fastapi logger (link)

Caching requests

fastapi background tsks

Examples

Как мы попробовали DDD, CQRS и Event Sourcing и какие выводы сделали (link)

CQRS (link)

Code examples

• https://github.com/Rustem/toptal-blog-celery-toy-ex
• https://github.com/arshadansari27/simple-analytics-engine/tree/cqrs
• https://github.com/iamsarthakjoshi/python-celery-rabbitmq
• https://github.com/szwang/celery-example
• https://github.com/rabbitmq/rabbitmq-tutorials/tree/main/python
• https://github.com/GregaVrbancic/fastapi-celery
• https://medium.com/@rijinswaminathan/use-celery-and-rabbitmq-with-django-rest-api-d803681d8c86
• https://medium.com/cuddle-ai/async-architecture-with-fastapi-celery-and-rabbitmq-c7d029030377
• https://github.com/tiangolo/full-stack-fastapi-postgresql
• https://github.com/tiangolo/full-stack-fastapi-couchbase

Models

MDF

• fix

Traceback (most recent call last):
  File "/app/app/main.py", line 47, in <module>
    main()
  File "/app/app/main.py", line 40, in main
    sentences = get_sentences(sources, model_class.model_id)
  File "/app/app/api.py", line 43, in get_sentences
    return [Sentence(**text) for text in response.json()["items"]]
  File "/usr/local/lib/python3.10/site-packages/requests/models.py", line 975, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Libs

Json

• https://github.com/ultrajson/ultrajson
• https://github.com/ijl/orjson

Docs

citar-export-local-bib-file

Спросить про то как обучалась модель и про сайт будет или нет