Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compare membership #13

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .tool-versions
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
erlang 21.2.6
elixir 1.8.1-otp-21
erlang 22.0.1
elixir 1.8.2-otp-22
12 changes: 7 additions & 5 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,11 @@ Thank you for contributing! Let's get you started!

1. Fork and clone the repository

2. Install dependencies: `mix deps.get`
1. Install the Elixir/Erlang versions in [`.tool-versions`](.tool-versions)

3. Write a benchmark using the following template:
1. Install dependencies: `mix deps.get`

1. Write a benchmark using the following template:

```elixir
defmodule IdiomName.Fast do
Expand Down Expand Up @@ -37,8 +39,8 @@ Thank you for contributing! Let's get you started!
IdiomName.Benchmark.benchmark()
```

4. Run your benchmark: `mix run code/<category>/<benchmark>.exs`
1. Run your benchmark: `mix run code/<category>/<benchmark>.exs`

5. Add the output along with a description to the [README](README.md).
1. Add the output along with a description to the [README](README.md)

6. Open a Pull Request!
1. Open a Pull Request!
64 changes: 64 additions & 0 deletions code/general/membership.exs
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
defmodule Membership.NewMapSetMember do
def check_membership(range, list, _mapset) do
find = Enum.random(range)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fairly certain that the runtime of Enum.random/1 should be constant, but for the sake of making these comparisons even stronger, how would you feel about moving this call to a before_each hook so the runtime of Enum.find/1 isn't included in the comparison?

mapset = MapSet.new(list)
MapSet.member?(mapset, find)
end
end

defmodule Membership.MapSetMember do
def check_membership(range, _list, mapset) do
find = Enum.random(range)
MapSet.member?(mapset, find)
end
end

defmodule Membership.EnumMember do
def check_membership(range, list, _mapset) do
find = Enum.random(range)
Enum.member?(list, find)
end
end

defmodule Membership.EnumAny do
def check_membership(range, list, _mapset) do
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure this is fair to include in this comparison since Enum.any?/2 is a much more flexible function than the others that we're comparing. It will almost always be slower since in the majority of use cases it's testing for more than just strict equality (i.e. comparing with Kernel.===/2 like all the other functions do).

find = Enum.random(range)
Enum.any?(list, &(&1 == find))
end
end

defmodule Membership.In do
def check_membership(range, list, _mapset) do
find = Enum.random(range)
find in list
end
end

defmodule Membership.Benchmark do
@inputs %{
"Large" => {1..1_000_000, Enum.shuffle(1..1_000_000), MapSet.new(1..1_000_000)},
"Medium" => {1..10_000, Enum.shuffle(1..10_000), MapSet.new(1..10_000)},
"Small" => {1..100, Enum.shuffle(1..100), MapSet.new(1..100)}
}

def benchmark do
Benchee.run(
%{
"New MapSet + MapSet.member?" => fn input -> bench(Membership.NewMapSetMember, input) end,
Copy link
Contributor Author

@dideler dideler May 27, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wasn't sure whether to keep this scenario. It ends up being the slowest for all inputs. Including may be useful to show that you lose all performance benefits when creating a new mapset that's infrequently used.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm very in favor of keeping this. Sometimes when folks look at stuff like this they think "oh, MapSet is faster, so I'll use that!" but in the end the added runtime of the conversion ends up making their code slower overall.

That's the unfortunate consequence of microbenchmarks. They can be helpful, but you also need to look at the macro picture.

"MapSet.member?" => fn input -> bench(Membership.MapSetMember, input) end,
"Enum.member?" => fn input -> bench(Membership.EnumMember, input) end,
"Enum.any?" => fn input -> bench(Membership.EnumAny, input) end,
"x in y" => fn input -> bench(Membership.In, input) end
},
time: 10,
inputs: @inputs,
print: [fast_warning: false]
)
end

defp bench(module, {range, list, mapset}) do
module.check_membership(range, list, mapset)
end
end

Membership.Benchmark.benchmark()
33 changes: 10 additions & 23 deletions mix.exs
Original file line number Diff line number Diff line change
Expand Up @@ -2,31 +2,18 @@ defmodule FastElixir.Mixfile do
use Mix.Project

def project do
[app: :fast_elixir,
version: "0.1.0",
elixir: "~> 1.4",
build_embedded: Mix.env == :prod,
start_permanent: Mix.env == :prod,
deps: deps()]
[
app: :fast_elixir,
version: "0.1.0",
elixir: "~> 1.6",
build_embedded: Mix.env() == :prod,
start_permanent: Mix.env() == :prod,
deps: deps()
]
end

# Configuration for the OTP application
#
# Type "mix help compile.app" for more information
def application do
[applications: [:logger]]
end

# Dependencies can be Hex packages:
#
# {:mydep, "~> 0.3.0"}
#
# Or git/path repositories:
#
# {:mydep, git: "https://github.com/elixir-lang/mydep.git", tag: "0.1.0"}
#
# Type "mix help deps" for more examples and options
# Type "mix help deps" for more information
defp deps do
[{:benchee, "~> 0.14"}]
[{:benchee, "~> 1.0"}]
end
end
4 changes: 2 additions & 2 deletions mix.lock
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
%{
"benchee": {:hex, :benchee, "0.14.0", "f771f587c48b4824b497e2a3e374f75e93ef01fc329873b089a3f5dd961b80b8", [:mix], [{:deep_merge, "~> 0.1", [hex: :deep_merge, repo: "hexpm", optional: false]}], "hexpm"},
"deep_merge": {:hex, :deep_merge, "0.2.0", "c1050fa2edf4848b9f556fba1b75afc66608a4219659e3311d9c9427b5b680b3", [:mix], [], "hexpm"},
"benchee": {:hex, :benchee, "1.0.1", "66b211f9bfd84bd97e6d1beaddf8fc2312aaabe192f776e8931cb0c16f53a521", [:mix], [{:deep_merge, "~> 1.0", [hex: :deep_merge, repo: "hexpm", optional: false]}], "hexpm"},
"deep_merge": {:hex, :deep_merge, "1.0.0", "b4aa1a0d1acac393bdf38b2291af38cb1d4a52806cf7a4906f718e1feb5ee961", [:mix], [], "hexpm"},
"poison": {:hex, :poison, "3.1.0", "d9eb636610e096f86f25d9a46f35a9facac35609a7591b3be3326e99a0484665", [:mix], []},
}