Thread parse script #117
Closed
Thread parse script #117
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Updating main branch
* origin/main: (136 commits) test(helpers/validate): mix format style(helpers/validate): mix format fix(helpers/validate): correctly implement mutually_exclusive test(helpers/validate): test mutually exclusive with several cases test(helpers/validate): move docs to test (for testing later) docs(helpers/validate): fix spelling errors in docs test(helper/validate): add doctest for mutually_exclusive! function refactor(controller/post): update comment todos feat(helpers/validate): add function to check mutual exclusivity of body parameters feat(errors/custom_errors): allow custom message for invalid payload error test(controllers/board): use head tail instead of Enum.at for checking response boards test(controller/board): refactor test descriptions for movelist tests(controllers/board): fix view order style(format): run mix format test(controllers/board): check that movelist obeys priority feat(controllers/post): add mentions hook to post by thread feat(portroadmap): update document containing port progress feat(controller/board): add route for board movelist feat(models/boards): implement movelist database function refactor(models/boards): wip implement movelist db function ...
Updating Main Branch
* origin/main: (120 commits) refactor(controllers/thread): use new function headers for parse and sanitize, this will fix editing of threads refactor(controllers/post): use new function headers for parse and sanitize, this will fix editing of posts refactor(helpers/parse): update parse function to store parse result into body_html instead of body refactor(helpers/sanitize): update sanitize functions to take in only attrs, store santized body in body_html refactor(models/post): update todo list for post create refactor(controllers): update post and thread controller to use update sanitize helper * remove unused html_sanitize_ex module refactor(helpers/sanitize): update html_from_body to html_and_entities_from_body, remove html sanitizer, use regex feat(helpers/sanitize): update strip from title and subject to only strip entities and lt and gt style(format): run mix format refactor(post): update how post are created, use body_html feat(helpers/parse): helper for parsing threads and posts feat(controllers): use md parser when creating posts and threads feat(dependencies/earmark): add earmark as markdown parser docs(helpers/sanitize): update docs to reflect function name change refactor(helpers/sanitizer): change function naming standard test(controllers/notification): resolve issue race condition with notification test setup feat(controller): use sanitizer in thread and post controllers test(helpers/sanitize): add doctest for sanitize helper functions feat(helpers/sanitize): function for sanitizing message subject feat(controllers/thread): strip html from thread title ...
…dded sequences to setup DB with IDs after the proxy DB IDs
…troller and by_thread route in posts controller to display posts on the frontend
…d and posts_by_thread through proxy
BTCT Proxy
BTCT Proxy Format
this project actually loads production configs from config/runtime.exs
Runtime Config
* origin/main: fix(runtime): show sensitive data set to false
…g smfrepo into application.ex
SmfRepo Proxy Supervisor Change
… avatar field from proxy data pull
Removed GenServer code and avatar field; Added is_proxy flag
feat(helpers/proxy_conversion): load board last post user avatar
specifically, the poolboy_config for bbc_parser
10 seconds is much too long, we don't want to take up resources for that amount of time. 1 second is fairly long as well
User find
* origin/main: (36 commits) style(application): mix format refactor(bbc_parser): reduce bbc parser php call timeout to 1 second refactor(config): poolboy_config -> bbc_parser_poolboy_config feat(helpers/proxy_conversion): load board last post user avatar feat(docker): upgrade elixir version in docker file fix(credo): resolve credo error with user_json refactor(user-find): move manipulation of user object into user_json file fix(last-login): use users last login instead of test value, resolve credo error feat(last-active): implement last active, respecting flag showOnline if user doesnt want to be shown as active within the past 72 hours test(warnings): resolve test warnings feat(user-find): convert gender and dob into human readable format test(static): resolve dialyzer errors after upgrade of elixir/erlang feat(dob): calculate dob for user find proxy feat(gender): calculate gender for user find proxy refactor(cleanup): use BBCParser Gen Server for parsing user signature, run format feat(last-active): bring back last post date for last active in user find fix(credo): resolve credo issues in bbc parser feat(credo): upgrade credo to latest fix(parser): add missing settings that were causing portuguese characters to break feat(parser): user async parser for signature, code cleanup ...
Board Moderators
…and none with a nil id_member;
Moderators Fix
this function is not async
return original data if parsing failed, instead of :ok (confusing to look at in UI) improve logging * user Logger.error * log error message, type, pid * specify module/function better
set a timeout for porcelain php parser call
differentiate call and receive timeouts
return {:ok, parsed} when data is parsed in time
return {:timeout, unparsed} when data does not return in time
handle return types after genserver call
log as error when timeout happens
this will help us debug and find offending bbcode in logs
…ing parser to fail
fixes dialyzer unreachable case
…d log when there is an error
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.