Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Posts by thread #71

Merged
merged 121 commits into from
Oct 26, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
121 commits
Select commit Hold shift + click to select a range
1915e8b
fix(models/auto_moderation): schema field types for auto_moderation m…
akinsey Sep 13, 2023
997fda8
refactor(models/auto_moderation): wip implement automoderation algorithm
akinsey Sep 16, 2023
184e9b2
refactor(models/auto_moderation): wip implement automoderation algorithm
akinsey Sep 18, 2023
8dbd8ae
Merge remote-tracking branch 'origin/main' into posts-by-thread
akinsey Sep 18, 2023
4017932
refactor(models/auto_moderation): wip implement moderate automoderati…
akinsey Sep 18, 2023
4f9a827
refactor(models/auto_moderation): wip implement moderate automoderati…
akinsey Sep 19, 2023
b30d0f8
refactor(models/auto_moderation): wip implement execute rule actions
akinsey Sep 20, 2023
7a5855b
feat(models/auto_moderation): initial implementation of auto moderate…
akinsey Sep 21, 2023
86926ed
feat(controllers/post): use auto moderator
akinsey Sep 22, 2023
1c9a441
fix(models/auto_moderation): unused variable
akinsey Sep 22, 2023
8fb1483
Merge remote-tracking branch 'origin/main' into posts-by-thread
akinsey Sep 22, 2023
22c8e0f
fix(models/posts): issue with locked and deleted not being set when a…
akinsey Sep 25, 2023
43813cb
fix(json): update json files for post and thread to properly handle d…
akinsey Sep 26, 2023
f5e4d0e
fix(plugs/prepare_parse): use plug builder to prevent issue with Post…
akinsey Sep 26, 2023
5cfd473
feat(migrations/posts): add tsv column to posts table
akinsey Sep 26, 2023
3bb8c2f
feat(posts): add trigger for posts tsv
akinsey Sep 27, 2023
2d197ed
feat(plugs/track_ip): implement plug that tracks user ip when making …
akinsey Sep 29, 2023
aa6c495
refactor(plugs/track_ip): format code and check if user is authentica…
akinsey Sep 29, 2023
3aee9b2
fix(board_json): clicking on a board errors
unenglishable Sep 29, 2023
9baff84
fix(metric_rank_map): default empty list when metric rank maps not po…
unenglishable Sep 29, 2023
340d52b
refactor(plugs/track_ip): update grammar, refactor syntax
akinsey Sep 30, 2023
82aee56
fix(controllers/user): register_with_verify, use base 16 instead of 64
unenglishable Sep 30, 2023
0cc0e0b
fix(user): implement maybe_confirm? on model, use in controller
unenglishable Sep 30, 2023
6967ceb
feat(plugins/last_active): add last active plugin
akinsey Oct 3, 2023
8333fac
refactor(models/mentions): wip implement mention hooks for post create
akinsey Oct 3, 2023
1944e04
refactor(models/mentions): wip implement username to user id helper.
akinsey Oct 4, 2023
974ceb2
style(models/user): run mix format
akinsey Oct 4, 2023
85e0237
feat(models/users): add way to query user_ids using list of usernames…
akinsey Oct 5, 2023
26372d6
feat(models/mention): initial implementation of username to user id f…
akinsey Oct 6, 2023
08ae9b4
refactor(session+auth): use `id` instead of `user_id` for guardian su…
unenglishable Oct 6, 2023
b62eee2
docs(session): update comment about jti
unenglishable Oct 6, 2023
3911d39
feat(session): after save, update guardian resource with session info
unenglishable Oct 6, 2023
7a48c90
refactor(models/user_ip): delete handler for old guardian default res…
unenglishable Oct 6, 2023
73699d1
test(user_socket): update Guardian.encode_and_sign, user_id -> id
unenglishable Oct 6, 2023
6d98f95
style(session): mix format
unenglishable Oct 6, 2023
eddd90f
Merge pull request #68 from epochtalk/session-resource-fix
akinsey Oct 6, 2023
af8170b
refactor(models/post): update correct tsv database call syntax to be …
akinsey Oct 6, 2023
02024ef
feat(models/mention): add function skeleton for handle_user_mention_c…
akinsey Oct 9, 2023
3682830
feat(models/mention): add helper function to correct text search vect…
akinsey Oct 10, 2023
01d75d3
feat(models/notifications): add function to handle creation of notifi…
akinsey Oct 10, 2023
c88d4b7
refactor(models/mention): wip implement handle_user_mention_creation
akinsey Oct 10, 2023
d414669
feat(mailer): add email for mention notification
akinsey Oct 10, 2023
7085e19
feat(models/user): get user email by id, for use with mentions
akinsey Oct 11, 2023
5c43fd4
fix(mailer): thread url format and email to field
akinsey Oct 12, 2023
22812ec
feat(models/thread): implement get_first_post_data_by_id for use with…
akinsey Oct 12, 2023
1338d20
feat(models/mention): complete handle_user_mention_creation function,…
akinsey Oct 12, 2023
03335e3
feat(models/user): implement new db function for fetching username by id
akinsey Oct 13, 2023
052133d
feat(models/mention): initial implementation of user_id_to_username
akinsey Oct 13, 2023
50cc843
fix(models/mention): close open parens
unenglishable Oct 13, 2023
11471ac
docs(models/mention): delete double quote
unenglishable Oct 13, 2023
3313312
refactor(models/mention): extend pipe chain
unenglishable Oct 13, 2023
301bd0d
feat(models/mention): update @username_mention_regex
unenglishable Oct 13, 2023
f9c8e03
feat(models/mention): add curly version of username mention regex
unenglishable Oct 14, 2023
002d3e6
refactor(models/mention): remove unused conn from user_id_to_username
unenglishable Oct 14, 2023
63cef0d
fix(models/mention): use profile_link user_id_to_username, from paste…
unenglishable Oct 14, 2023
1685917
refactor(models/mention): username_to_user_id, accept user instead of…
unenglishable Oct 14, 2023
bb1b858
refactor(models/mention): pass user instead of conn to handle_user_me…
unenglishable Oct 14, 2023
6ab8c2f
feat(controllers/post): integrate mentions in posts controller
unenglishable Oct 14, 2023
982effc
docs(models/mention): remove todo for code blocks
unenglishable Oct 14, 2023
07b3310
feat(models/mention): update username_to_user_id, handle "possible_us…
unenglishable Oct 14, 2023
ef507f6
style(): mix format
unenglishable Oct 16, 2023
d3ce368
Merge pull request #70 from epochtalk/mentions-regex
akinsey Oct 16, 2023
ca978a7
fix(models/mention): resolve credo error caused by over nesting user_…
akinsey Oct 16, 2023
d3cd2a9
refactor(controllers/post): update comments
akinsey Oct 16, 2023
ed33587
feat(controllers/thread): add hook for watched board to threads by bo…
akinsey Oct 16, 2023
e7fed60
refactor(models/thread): add comment with roadmap for finishing port …
akinsey Oct 17, 2023
a303ad0
refactor(models/thread): add authorizations todos
akinsey Oct 17, 2023
d3ac000
feat(regex): add regex.ex
unenglishable Oct 17, 2023
95ead40
test(regex): add test for regex
unenglishable Oct 17, 2023
0bed58c
test(regex): refactor test names to match guidelines
unenglishable Oct 17, 2023
c53a0de
test(regex): add test string attribute
unenglishable Oct 17, 2023
a158c2b
test(regex): test regex scan for username_mention
unenglishable Oct 17, 2023
31b1717
test(regex): use Enum.each for checking usernames
unenglishable Oct 17, 2023
5fd9379
test(regex): extract usernames from test, into attribute
unenglishable Oct 17, 2023
fd1d540
refactor(controllers/thread): wip implement authorizations for thread…
akinsey Oct 17, 2023
31dae2e
test(regex): check :username_mention_curly
unenglishable Oct 17, 2023
16fd728
test(regex): mix format
unenglishable Oct 17, 2023
cffe578
test(regex): check that mention can have a trailing comma
unenglishable Oct 17, 2023
07857c1
test(regex): test mixed case mention
unenglishable Oct 17, 2023
7e5f4ca
refactor(controllers/thread): add user active check to thread create,…
akinsey Oct 17, 2023
7e84f5a
fix(regex): user_id; remove bol from pattern, use 0-9 instead of [:di…
unenglishable Oct 17, 2023
0d45e9e
feat(models/board): implement new db function for thread create autho…
akinsey Oct 18, 2023
159d756
refactor(models/poll): add comment for future task
akinsey Oct 18, 2023
8a0a2a9
feat(helpers/acl): allow user to be a map for allow! function
akinsey Oct 18, 2023
3486a18
test(regex): create test for :user_id pattern
unenglishable Oct 19, 2023
0fe5d03
test(regex): split test for mentions out
unenglishable Oct 19, 2023
4bdf2d3
test(regex): refactor module attribute names for mentions test
unenglishable Oct 19, 2023
2364a24
test(regex): mix format
unenglishable Oct 19, 2023
4659d9c
test(regex): move attributes near mentions test
unenglishable Oct 19, 2023
f226160
test(regex): remove unnecessary id_to_username_map
unenglishable Oct 19, 2023
b410f31
refactor(models/mention): use EpochtalkServer.Regex.pattern
unenglishable Oct 19, 2023
ca36d67
test(models/mention): add mention test
unenglishable Oct 19, 2023
26e4cea
test(models/mention): username_to_user_id/2
unenglishable Oct 19, 2023
d0cb639
test(models/mention): test valid mentions get generated
unenglishable Oct 19, 2023
1f31baa
test(models/mention): check that all user ids are present in mentione…
unenglishable Oct 19, 2023
6396f3e
test(models/mention): check behavior on empty user
unenglishable Oct 19, 2023
100daec
test(models/mention): refactor invalid user body
unenglishable Oct 19, 2023
3322c65
test(models/mention): test invalid thread id
unenglishable Oct 19, 2023
9dd102a
test(models/mention): test empty body
unenglishable Oct 19, 2023
c60f356
test(models/mention): test user without acl permission
unenglishable Oct 19, 2023
7effba3
test(models/mention): add note for duplicate mention
unenglishable Oct 19, 2023
e76818f
test(models/mention): test empty title
unenglishable Oct 19, 2023
75219eb
style(models/mention): mix format
unenglishable Oct 19, 2023
e53e4c7
test(models/mention): mix format
unenglishable Oct 19, 2023
7cd2c25
refactor(models/threads): update thread create to take user instead o…
akinsey Oct 19, 2023
2dff9d4
refactor(models/thread): update handle_create_post parameter order fo…
akinsey Oct 19, 2023
5cc3d04
feat(controllers/post): check if user is banned from board before cre…
akinsey Oct 20, 2023
08036f4
feat(controllers/thread): check if user is board banned and that they…
akinsey Oct 20, 2023
fe706af
feat(models/poll): implement changeset validator to check for display…
akinsey Oct 20, 2023
6dfba59
refactor(models/polls): mix format)
akinsey Oct 20, 2023
603ddfd
refactor(controllers/thread): run mix format
akinsey Oct 20, 2023
c7227f6
fix(tests/thread): fix thread create factory
akinsey Oct 21, 2023
015efe2
feat(controllers/thread): finish porting hooks for thread create rout…
akinsey Oct 23, 2023
a93c43d
fix(dialyzer): resolve dialyzer error caused by fetching post out of …
akinsey Oct 23, 2023
8424f8d
refactor(json/thread_json): update comment
akinsey Oct 23, 2023
b3547ef
Merge pull request #72 from epochtalk/mentions-regex
akinsey Oct 24, 2023
90006e6
fix(tests/mentions): resolve issue with mentions tests failing after …
akinsey Oct 24, 2023
d545e2c
refactor(auto_moderation): clarify module attribute names
unenglishable Oct 26, 2023
20afd1c
docs(models/mention): spelling
unenglishable Oct 26, 2023
075a6bf
refactor(controllers/thread): update errors for specific cases
unenglishable Oct 26, 2023
3af12b7
style(): mix format
unenglishable Oct 26, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion lib/epochtalk_server/auth/guardian.ex
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,7 @@ defmodule EpochtalkServer.Auth.Guardian do
The subject should be a short identifier that can be used to identify
the resource.
"""
def subject_for_token(%{user_id: user_id}, _claims) do
def subject_for_token(%{id: user_id}, _claims) do
# You can use any value for the subject of your token but
# it should be useful in retrieving the resource later, see
# how it being used on `resource_from_claims/1` function.
Expand Down
41 changes: 39 additions & 2 deletions lib/epochtalk_server/mailer.ex
Original file line number Diff line number Diff line change
Expand Up @@ -47,10 +47,8 @@ defmodule EpochtalkServer.Mailer do
email: email,
id: last_post_id,
position: last_post_position,
thread_author: _thread_author,
thread_slug: thread_slug,
title: thread_title,
user_id: _user_id,
username: username
}) do
config = Application.get_env(:epochtalk_server, :frontend_config)
Expand Down Expand Up @@ -81,6 +79,45 @@ defmodule EpochtalkServer.Mailer do
|> handle_delivered_email()
end

@doc """
Sends mention notification email
"""
@spec send_mention_notification(email_data :: map) :: {:ok, term} | {:error, term}
def send_mention_notification(%{
email: email,
post_id: post_id,
post_position: post_position,
post_author: post_author,
thread_slug: thread_slug,
thread_title: thread_title
}) do
config = Application.get_env(:epochtalk_server, :frontend_config)
frontend_url = config["frontend_url"]
website_title = config["website"]["title"]
from_address = config["emailer"]["options"]["from_address"]

thread_url = "#{frontend_url}/threads/#{thread_slug}?start=#{post_position}##{post_id}"

content =
generate_from_base_template(
"""
<h3>#{post_author} mentioned you in the thread "#{thread_title}"</h3>
Please visit the link below to view the post you were mentioned in.<br /><br />
<a href="#{thread_url}">View Mention</a>
<small>Raw thread URL: #{thread_url}</small>
""",
config
)

new()
|> to(email)
|> from({website_title, from_address})
|> subject("[#{website_title}] New replies to thread #{thread_title}")
|> html_body(content)
|> deliver()
|> handle_delivered_email()
end

defp handle_delivered_email(result) do
case result do
{:ok, email_metadata} ->
Expand Down
236 changes: 234 additions & 2 deletions lib/epochtalk_server/models/auto_moderation.ex
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,16 @@ defmodule EpochtalkServer.Models.AutoModeration do
import Ecto.Changeset
import Ecto.Query
alias EpochtalkServer.Repo
alias EpochtalkServer.Session
alias EpochtalkServer.Models.AutoModeration
alias EpochtalkServer.Models.Ban
alias EpochtalkServerWeb.CustomErrors.AutoModeratorReject

@postgres_varchar255_max 255
@postgres_varchar1000_max 1000
@hours_per_day 24
@minutes_per_hour 60
@seconds_per_minute 60

@moduledoc """
`AutoModeration` model, for performing actions relating to `User` `AutoModeration`
Expand Down Expand Up @@ -38,8 +44,8 @@ defmodule EpochtalkServer.Models.AutoModeration do
field :name, :string
field :description, :string
field :message, :string
field :conditions, :map
field :actions, :map
field :conditions, {:array, :map}
field :actions, {:array, :string}
field :options, :map
field :created_at, :naive_datetime
field :updated_at, :naive_datetime
Expand Down Expand Up @@ -149,4 +155,230 @@ defmodule EpochtalkServer.Models.AutoModeration do
|> update_changeset(auto_moderation_attrs)
|> Repo.update()
end

## === Public Helper Functions ===

@doc """
Executes `AutoModeration` rules

TODO(akinsey): Optimize and store rules in Redis so we dont have to query every request

### Rule Anatomy
* Only works on posts
* = Name: Name for this rule (for admin readability)
* = Description: What this rule does (for admin readbility)
* = Message: Error reported back to the user on reject action
* = Conditions: condition regex will only work on
* - body
* - thread_id
* - user_id
* - title (although it's not much use)
* == REGEX IS AN OBJECT with a pattern and flag property
* Multiple conditions are allow but they all must pass to enable rule actions
* = Actions: reject, ban, edit, delete (filter not yet implemented)
* = Options:
* - banInterval:
* - Affects ban action.
* - Leave blank for permanent
* - Otherwise, JS date string
* - edit:
* - replace (replace chunks of text):
* - regex: Regex used to match post body
* - regex object has a pattern and flag property
* - text: Text used to replace any matches
* - template: String template used to add text above or below post body
"""
@spec moderate(user :: map, post_attrs :: map) :: post_attrs :: map
def moderate(%{id: user_id} = _user, post_attrs) do
# query auto moderation rules from the db, check their validity then return them
rule_actions = get_rule_actions(post_attrs)

# append user_id to post attributes
post_attrs = post_attrs |> Map.put("user_id", user_id)

# execute rule actions if action set isn't empty, then return updated post_attributes
post_attrs =
if MapSet.size(rule_actions.action_set) > 0,
do: execute_rule_actions(post_attrs, rule_actions),
else: post_attrs

# return updated post attributes
post_attrs
end

## === Private Helper Functions ===

defp get_rule_actions(post_attrs) do
acc_init = %{
action_set: MapSet.new(),
messages: [],
ban_interval: nil,
edits: []
}

rules = AutoModeration.all()

Enum.reduce(rules, acc_init, fn rule, acc ->
if rule_condition_is_valid?(post_attrs, rule.conditions) do
# Aggregate all actions, using MapSet ensures actions are unique
action_set = (MapSet.to_list(acc.action_set) ++ rule.actions) |> MapSet.new()

# Aggregate all reject messages if applicable
messages =
if Enum.member?(rule.actions, "reject") and is_binary(rule.message),
do: acc.messages ++ [rule.message],
else: acc.messages

# attempt to set default value for acc.ban_interval if nil
acc =
if is_nil(acc.ban_interval),
do: Map.put(acc, :ban_interval, rule.options["ban_interval"]),
else: acc

# Pick the latest ban interval, in the event multiple are provided
ban_interval =
if Enum.member?(rule.actions, "ban") and
Map.has_key?(rule.options, "ban_interval") and
acc.ban_interval < rule.options["ban_interval"],
do: rule.options["ban_interval"],
else: acc.ban_interval

# Aggregate all edit options
edits =
if Enum.member?(rule.actions, "edit"),
do: acc.edits ++ [rule.options["edit"]],
else: acc.edits

# return updated acc
%{
action_set: action_set,
messages: messages,
ban_interval: ban_interval,
edits: edits
}
else
acc
end
end)
end

defp execute_rule_actions(
post_attrs,
%{
action_set: action_set,
messages: messages,
ban_interval: ban_interval,
edits: edits
} = _rule_actions
) do
# handle rule actions that edit the post body
post_attrs =
if MapSet.member?(action_set, "edit"),
do:
Enum.reduce(edits, post_attrs, fn edit, acc ->
post_body = acc["body"]

# handle actions that replace text in post body
acc =
if is_map(edit["replace"]) do
replacement_text = edit["replace"]["text"]
test_pattern = edit["replace"]["regex"]["pattern"]
test_flags = edit["replace"]["regex"]["flags"]

# compensate for elixir not supporting /g/ flag
replace_globally = String.contains?(test_flags, "g")

# remove g flag (doesnt work in elixir), compensate later using string replace
test_flags = Regex.replace(~r/g/, test_flags, "")
match_regex = Regex.compile!(test_pattern, test_flags)

# update body of post with replacement text
updated_post_body =
String.replace(post_body, match_regex, replacement_text,
global: replace_globally
)

# return acc with updated post body
Map.put(acc, "body", updated_post_body)
else
acc
end

# handle actions that replace post body using a template
acc =
if is_binary(edit["template"]) do
# get new post body template
template = edit["template"]

# update post body using template
updated_post_body = String.replace(template, "{body}", post_body)

# return post_attrs with updated post body
Map.put(acc, "body", updated_post_body)
else
acc
end

# return updated acc
acc
end),
else: post_attrs

# handle rule actions that ban the user
if MapSet.member?(action_set, "ban") do
# ban period is utc now plus how ever many days ban_interval is
ban_period =
if ban_interval,
do:
DateTime.utc_now()
|> DateTime.add(
ban_interval * @hours_per_day * @minutes_per_hour * @seconds_per_minute,
:second
)
|> DateTime.to_naive()

# get user_id from post_attrs
user_id = post_attrs["user_id"]

# ban the user, ban_period is either a date or nil (permanent)
Ban.ban_by_user_id(user_id, ban_period)

# update user session after banning
Session.update(user_id)

# send websocket notification to reauthenticate user
EpochtalkServerWeb.Endpoint.broadcast("user:#{user_id}", "reauthenticate", %{})
end

# handle rule actions that shadow delete the post (auto lock/delete)
post_attrs =
if MapSet.member?(action_set, "delete"),
do: post_attrs |> Map.put("deleted", true) |> Map.put("locked", true),
else: post_attrs

# handle rule actions that reject the post entirely
if MapSet.member?(action_set, "reject"),
do:
raise(AutoModeratorReject,
message: "Post rejected by Auto Moderator: #{Enum.join(messages, ", ")}"
)

post_attrs
end

defp rule_condition_is_valid?(post_attrs, conditions) do
matches =
Enum.map(conditions, fn condition ->
test_param = post_attrs[condition["param"]]
test_pattern = condition["regex"]["pattern"]
test_flags = condition["regex"]["flags"]
# remove g flag, one match is good enough to determine if condition is valid
test_flags = Regex.replace(~r/g/, test_flags, "")
match_regex = Regex.compile!(test_pattern, test_flags)
Regex.match?(match_regex, test_param)
end)

# Only valid if every condition returns a valid regex match
!Enum.member?(matches, false)
end
end
19 changes: 19 additions & 0 deletions lib/epochtalk_server/models/board.ex
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,25 @@ defmodule EpochtalkServer.Models.Board do
end
end

@doc """
Given an id and moderated property, returns boolean indicating if `Board` allows self moderation
"""
@spec allows_self_moderation?(id :: non_neg_integer, moderated :: boolean) ::
allowed :: boolean
def allows_self_moderation?(id, true) do
query =
from b in Board,
where: b.id == ^id,
select: b.meta["disable_self_mod"]

query
|> Repo.one() || false
end

# pass through if model being created doesn't have "moderated" property set
def allows_self_moderation?(_id, false), do: true
def allows_self_moderation?(_id, nil), do: true

@doc """
Determines if the provided `user_priority` has write access to the board that contains the thread
the specified `thread_id`
Expand Down
Loading
Loading