Added FORMAT option to ASSERT_EQUAL (#77)

Allows buffers to be compared in a manner such that messages about
mismatching elements are presented at a higher level of granularity
than the byte level, and also supports skipping elements (e.g. to skip
padding).

Fixes #75.

docs/shadertrap_language.md

@@ -54,11 +54,31 @@ A brief description of each ShaderTrap command now follows. The commands are pre
 This command has two forms.
 
 ```
-ASSERT_EQUAL BUFFERS buffer_1 buffer_2
+ASSERT_EQUAL BUFFERS buffer_1 buffer_2 [FORMAT format_entry_1 ... format_entry_n]
 ```
 
 Checks whether `buffer_1` and `buffer_2`, which must be buffers produced by `CREATE_BUFFER`, contain equal contents. Performs a byte-level comparison of the buffers. Yields an error if the buffers have different sizes, or if they have the same size but differ at any single byte.
 
+- `buffer_1` and `buffer_2` are the buffers to be compared
+- Each `format_entry_i` is either:
+    - `SKIP_BYTES count` for some positive integer `count` that must be a multiple of 4
+    - `type count` for some positive integer `count`, where `type` is one of `byte`, `int`, `uint` or `float`, and where `count` is a multiple of 4 if `type` is `byte`
+
+`FORMAT` is an optional parameter that specifies the format of the data that will be compared. If `FORMAT` is specified, it should be followed by at least 1 `format_entry_i` component. The data is then  formatted by processing the `format_entry_i` components as follows, with respect to `offset`, a byte offset into the buffer, initialised to 0:
+
+- If `format_entry_i` is `SKIP_BYTES count` then `offset` is incremented by `count`. This allows padding or irrelevant data to be ignored.
+
+- If `format_entry_i` is byte count then count successive bytes from the buffers are compared, starting from position offset, with any byte-level differences reported, after which offset is incremented by `count`.
+
+- If `format_entry_i` is int count then count successive 4-byte sequences from the buffers are compared, starting from position offset, after which offset is incremented by 4* `count`. Any mismatches between 4-byte sequences are reported as differences between 32-bit signed integers.
+
+- If format_entry_i is `uint count` then the effect is the same as for the `int` case, except that mismatches are reported as differences between 32-bit unsigned integers.
+
+- If `format_entry_i` is `float count` then the effect is the same as for the `int` case, except that mismatches are reported as differences between 32-bit floating-point values. Note that this does not involve comparing floating-point numbers: comparisons are made at the byte level.
+
+The sum of `count` for all `byte` and `SKIP_BYTES` entries, plus the sum of 4*`count` for all `int`, `uint` and `float` entries, must equal the buffer size in bytes - i.e., every byte in the buffer must be accounted for.
+
+If `FORMAT` is not explicitly specified in the command we assumed it to be `FORMAT byte n` where `n` is the size of `buffer_1` and `buffer_2`.
 ```
 ASSERT_EQUAL RENDERBUFFERS renderbuffer_1 renderbuffer_2
 ```

examples/OpenGL45/assert_equal_examples/assert_equal1.shadertrap

@@ -0,0 +1,23 @@
+# Copyright 2021 The ShaderTrap Project Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the Licens
+
+GL 4.5
+
+# Valid example
+
+CREATE_BUFFER buf1 SIZE_BYTES 16 INIT_VALUES float 1.0 2.0 3.0 4.0
+
+CREATE_BUFFER buf2 SIZE_BYTES 16 INIT_VALUES float 1.0 2.0 3.0 4.0
+
+ASSERT_EQUAL BUFFERS buf1 buf2 FORMAT float 4

examples/OpenGL45/assert_equal_examples/assert_equal2.shadertrap

@@ -0,0 +1,23 @@
+# Copyright 2021 The ShaderTrap Project Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+GL 4.5
+
+# Skips the first 2 unequal values.
+
+CREATE_BUFFER buf1 SIZE_BYTES 16 INIT_VALUES float 5.0 6.0 3.0 4.0
+
+CREATE_BUFFER buf2 SIZE_BYTES 16 INIT_VALUES float 1.0 2.0 3.0 4.0
+
+ASSERT_EQUAL BUFFERS buf1 buf2 FORMAT SKIP_BYTES 8 float 2

examples/OpenGL45/assert_equal_examples/assert_equal3.shadertrap

@@ -0,0 +1,23 @@
+# Copyright 2021 The ShaderTrap Project Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+GL 4.5
+
+# Skips unequal values in between equal ones.
+
+CREATE_BUFFER buf1 SIZE_BYTES 16 INIT_VALUES float 1.0 2.0 5.0 4.0
+
+CREATE_BUFFER buf2 SIZE_BYTES 16 INIT_VALUES float 1.0 2.0 3.0 4.0
+
+ASSERT_EQUAL BUFFERS buf1 buf2 FORMAT float 2 SKIP_BYTES 4 float 1

src/libshadertrap/include/libshadertrap/command_assert_equal.h

@@ -15,8 +15,10 @@
 #ifndef LIBSHADERTRAP_COMMAND_ASSERT_EQUAL_H
 #define LIBSHADERTRAP_COMMAND_ASSERT_EQUAL_H
 
+#include <cstddef>
 #include <memory>
 #include <string>
+#include <vector>
 
 #include "libshadertrap/command.h"
 #include "libshadertrap/token.h"
@@ -25,8 +27,20 @@ namespace shadertrap {
 
 class CommandAssertEqual : public Command {
  public:
+  struct FormatEntry {
+    enum class Kind { kByte, kFloat, kInt, kUint, kSkip };
+    std::unique_ptr<Token> token;
+    Kind kind;
+    size_t count;
+  };
+  // Constructor used for an assertion about the equality of two buffers.
+  CommandAssertEqual(std::unique_ptr<Token> start_token,
+                     std::unique_ptr<Token> argument_identifier_1,
+                     std::unique_ptr<Token> argument_identifier_2,
+                     std::vector<FormatEntry> format_entries);
+
+  // Constructor used for an assertion about the equality of two renderbuffers.
   CommandAssertEqual(std::unique_ptr<Token> start_token,
-                     bool arguments_are_renderbuffers,
                      std::unique_ptr<Token> argument_identifier_1,
                      std::unique_ptr<Token> argument_identifier_2);
 
@@ -52,11 +66,14 @@ class CommandAssertEqual : public Command {
     return *argument_identifier_2_;
   }
 
+  std::vector<FormatEntry>& GetFormatEntries() { return format_entries_; }
+
  private:
   // true if arguments are renderbuffers, false if arguments are buffers
   bool arguments_are_renderbuffers_;
   std::unique_ptr<Token> argument_identifier_1_;
   std::unique_ptr<Token> argument_identifier_2_;
+  std::vector<FormatEntry> format_entries_;
 };
 
 }  // namespace shadertrap

src/libshadertrap/include/libshadertrap/parser.h

@@ -20,6 +20,7 @@
 #include <functional>
 #include <map>
 #include <memory>
+#include <set>
 #include <string>
 #include <utility>
 #include <vector>
@@ -62,7 +63,8 @@ class Parser {
 
   bool ParseParameters(
       const std::map<Token::Type, std::function<bool()>>& parameter_parsers,
-      const std::map<Token::Type, Token::Type>& mutually_exclusive);
+      const std::map<Token::Type, Token::Type>& mutually_exclusive,
+      const std::set<Token::Type>& optional_params);
 
   bool ParseParameters(
       const std::map<Token::Type, std::function<bool()>>& parameter_parsers);

src/libshadertrap/src/checker.cc

@@ -187,10 +187,66 @@ bool Checker::VisitAssertEqual(CommandAssertEqual* command_assert_equal) {
               std::to_string(buffer1->GetSizeBytes()) + " of '" +
               operand1_token.GetText() + "' at " +
               operand1_token.GetLocationString());
-      return false;
+      found_errors = true;
     }
   }
-  return true;
+  auto& format_entries = command_assert_equal->GetFormatEntries();
+  if (!format_entries.empty()) {
+    size_t total_count_bytes = 0;
+    for (const auto& format_entry : format_entries) {
+      if (format_entry.count == 0) {
+        message_consumer_->Message(
+            MessageConsumer::Severity::kError, format_entry.token.get(),
+            "The count for a formatting entry must be positive");
+        found_errors = true;
+      }
+      switch (format_entry.kind) {
+        case CommandAssertEqual::FormatEntry::Kind::kByte:
+        case CommandAssertEqual::FormatEntry::Kind::kSkip:
+          if (format_entry.count % 4 != 0) {
+            message_consumer_->Message(
+                MessageConsumer::Severity::kError, format_entry.token.get(),
+                "The count for a '" +
+                    Tokenizer::KeywordToString(
+                        format_entry.kind ==
+                                CommandAssertEqual::FormatEntry::Kind::kByte
+                            ? Token::Type::kKeywordTypeByte
+                            : Token::Type::kKeywordSkipBytes) +
+                    "' formatting entry must be a multiple of 4; found " +
+                    std::to_string(format_entry.count));
+            found_errors = true;
+          }
+          total_count_bytes += format_entry.count;
+          break;
+        case CommandAssertEqual::FormatEntry::Kind::kFloat:
+        case CommandAssertEqual::FormatEntry::Kind::kInt:
+        case CommandAssertEqual::FormatEntry::Kind::kUint:
+          total_count_bytes += format_entry.count * 4;
+          break;
+      }
+    }
+
+    auto* buffer1 = created_buffers_.at(operand1_token.GetText());
+    auto* buffer2 = created_buffers_.at(operand2_token.GetText());
+    const size_t expected_bytes = buffer1->GetSizeBytes();
+
+    if (total_count_bytes != expected_bytes) {
+      message_consumer_->Message(
+          MessageConsumer::Severity::kError,
+          command_assert_equal->GetFormatEntries()[0].token.get(),
+          "The number of bytes specified in the formatting of '" +
+              buffer1->GetResultIdentifier() + "(" +
+              buffer2->GetResultIdentifier() + ")" + "' is " +
+              std::to_string(total_count_bytes) + ", but '" +
+              buffer1->GetResultIdentifier() + "(" +
+              buffer2->GetResultIdentifier() + ")" +
+              "' was declared with size " + std::to_string(expected_bytes) +
+              " byte" + (expected_bytes > 1 ? "s" : "") + " at " +
+              buffer1->GetStartToken().GetLocationString());
+      found_errors = true;
+    }
+  }
+  return !found_errors;
 }
 
 bool Checker::VisitAssertPixels(CommandAssertPixels* command_assert_pixels) {

src/libshadertrap/src/command_assert_equal.cc

@@ -21,11 +21,22 @@
 namespace shadertrap {
 
 CommandAssertEqual::CommandAssertEqual(
-    std::unique_ptr<Token> start_token, bool arguments_are_renderbuffers,
+    std::unique_ptr<Token> start_token,
+    std::unique_ptr<Token> argument_identifier_1,
+    std::unique_ptr<Token> argument_identifier_2,
+    std::vector<FormatEntry> format_entries)
+    : Command(std::move(start_token)),
+      arguments_are_renderbuffers_(false),
+      argument_identifier_1_(std::move(argument_identifier_1)),
+      argument_identifier_2_(std::move(argument_identifier_2)),
+      format_entries_(std::move(format_entries)) {}
+
+CommandAssertEqual::CommandAssertEqual(
+    std::unique_ptr<Token> start_token,
     std::unique_ptr<Token> argument_identifier_1,
     std::unique_ptr<Token> argument_identifier_2)
     : Command(std::move(start_token)),
-      arguments_are_renderbuffers_(arguments_are_renderbuffers),
+      arguments_are_renderbuffers_(true),
       argument_identifier_1_(std::move(argument_identifier_1)),
       argument_identifier_2_(std::move(argument_identifier_2)) {}