-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Some fixes for the release blog post #17
base: master
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||||||
---|---|---|---|---|---|---|---|---|---|---|
|
@@ -13,41 +13,42 @@ categories: news | |||||||||
After a lot of time and effort, Kaitai project is happy to announce | ||||||||||
release of new major version of Kaitai Struct, declarative markup | ||||||||||
language to describe various binary data structures — binary file | ||||||||||
formats, network stream packets, etc. | ||||||||||
formats, network packets, etc. | ||||||||||
|
||||||||||
The basic idea of Kaitai Struct is that a particular format can be | ||||||||||
described using Kaitai Struct language (in a `.ksy` file), which then | ||||||||||
can be compiled using `kaitai-struct-compiler` into source files in | ||||||||||
one of the supported programming languages. These modules will include | ||||||||||
a generated code for a parser that can read described data structure | ||||||||||
from a file / stream and provide access to its contents in a nice, | ||||||||||
from a file / buffer and provide access to its contents in a nice, | ||||||||||
easy-to-comprehend API. | ||||||||||
|
||||||||||
With the previous 0.8 release, Kaitai project celebrated 1000 stars on GitHub, | ||||||||||
and until 0.9 version, it has collected more than [2000 stars](https://github.com/kaitai-io/kaitai_struct/stargazers). | ||||||||||
Thank you all for your support! | ||||||||||
|
||||||||||
This version introduces C++11 target (which uses smart pointers), | ||||||||||
several handy features (like `valid`ations and little-endian bit integers), | ||||||||||
fixes a lot of bugs and includes quite a few infrastructure improvements. | ||||||||||
several handy features (like `valid`ations and little-endian "bit-sized | ||||||||||
types"), fixes a lot of bugs and includes quite a few infrastructure | ||||||||||
improvements. | ||||||||||
|
||||||||||
## Release highlights | ||||||||||
|
||||||||||
* New targets support: | ||||||||||
* Python with [Construct](https://construct.readthedocs.io) library | ||||||||||
* HTML - intended for documentation, preliminary support | ||||||||||
* Nim - entry-level support (51% tests pass score) | ||||||||||
* New KSY language features: | ||||||||||
* Python with [Construct](https://construct.readthedocs.io) library ([#377](https://github.com/kaitai-io/kaitai_struct/issues/377)) | ||||||||||
* HTML - intended for documentation, preliminary support ([ec21b](https://github.com/kaitai-io/kaitai_struct_compiler/commit/ec21bdf62bcc9211e514dd8413ed5f6714a8e6bc)) | ||||||||||
* Nim - entry-level support (51% tests pass score) ([#619](https://github.com/kaitai-io/kaitai_struct/issues/619)) | ||||||||||
* New KS language features: | ||||||||||
generalmimon marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||||
* `doc-ref` supports list of references ([#269](https://github.com/kaitai-io/kaitai_struct/issues/269)) | ||||||||||
* `meta/tags` allows specification of multiple tags to allow better navigation in the format gallery ([#572](https://github.com/kaitai-io/kaitai_struct/issues/572)) | ||||||||||
* Allow accessing nested types using `::` syntax: `foo::bar` ([#275](https://github.com/kaitai-io/kaitai_struct/issues/275)) | ||||||||||
* Implement parsed data validations using `valid` key ([#435](https://github.com/kaitai-io/kaitai_struct/issues/435)) | ||||||||||
* Implement compile-time `sizeof` and `bitsizeof` operators ([#84](https://github.com/kaitai-io/kaitai_struct/issues/84)) | ||||||||||
* Type-based: `sizeof<u4>`, `bitsizeof<b13>`, `sizeof<user_type>` | ||||||||||
* Value-based: `file_header._sizeof`, `flags._bitsizeof` (`file_header`, `flags` are fields defined in the current type) | ||||||||||
* Implement little-endian bit-sized integers ([docs](https://doc.kaitai.io/user_guide.html#bit-ints-le)) | ||||||||||
* Support choosing endianness using `le` / `be` suffix: `type: b12le`, `type: b1be` | ||||||||||
* Add `meta/bit-endian` key for selecting default bit endianness (`le` / `be`) | ||||||||||
* Implement little-endian-based bit-sized types ([docs](https://doc.kaitai.io/user_guide.html#bit-ints-le)) | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What about just using the established terminology from the User Guide (https://doc.kaitai.io/user_guide.html#bit-ints-le) to which you had no serious objections (just "some offtop")?
Suggested change
It would be also OK to use little-endian layout for example. Yeah, I'm still not a fan of bit-sized types, and I don't really understand how #17 (comment) justifies it. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The problem is that that feature is mainly not about integers in little-endian order. Concept of endianness makes sense only for the stuff that have the size of multiple of endianness granularity (8 bit currently) and is aligned with that granularity. And it is not restricted to integers, it is just that bits sequences can be processed as integers and in order to process them on CPUs they have to be treated this way. |
||||||||||
* Support choosing endianness of bit-sized types using `le` / `be` suffix: `type: b12le`, `type: b1be` | ||||||||||
* Add `meta/bit-endian` key for selecting default bit endianness (`le` / `be`) ([#155](https://github.com/kaitai-io/kaitai_struct/issues/155)) | ||||||||||
* Expression language: | ||||||||||
* Forced byte array and true array literals ([#371](https://github.com/kaitai-io/kaitai_struct/issues/371)) and | ||||||||||
empty typed array literals ([#372](https://github.com/kaitai-io/kaitai_struct/issues/372)) | ||||||||||
|
@@ -63,7 +64,7 @@ fixes a lot of bugs and includes quite a few infrastructure improvements. | |||||||||
* C++: add C++11 mode | ||||||||||
* Add `--cpp-standard` CLI option: pass `--cpp-standard 11` to enable C++11 mode (`98` is default) | ||||||||||
* C++11 target: | ||||||||||
* uses `#pragma once` (instead of `#ifndef FOO_H_` header guards) | ||||||||||
* uses `#pragma once` (instead of `#ifndef FOO_H_` header guards) ([25fb1](https://github.com/kaitai-io/kaitai_struct_compiler/commit/25fb1eee61d07bdc8b199445776836ce9a6606ef)) | ||||||||||
* uses `std::unique_ptr<foo>` for owning pointers, raw pointers `foo*` for non-owning | ||||||||||
* supports array literals | ||||||||||
* `--no-auto-read` implemented for C++ | ||||||||||
|
@@ -72,6 +73,7 @@ fixes a lot of bugs and includes quite a few infrastructure improvements. | |||||||||
* Runtime API changes: | ||||||||||
* Add exceptions `Validation{Not{Equal,AnyOf},{Less,Greater}Than,Expr}Error` inheriting from common ancestor `ValidationFailedError` - thrown on failed validations defined with `valid` or `contents` key ([#435](https://github.com/kaitai-io/kaitai_struct/issues/435)) | ||||||||||
* Add method `read_bits_int_le` for parsing little-endian bit-sized integers ([docs](https://doc.kaitai.io/user_guide.html#bit-ints-le)) | ||||||||||
* Allow third-party `process`ors to be used ([#457](https://github.com/kaitai-io/kaitai_struct/issues/457)). | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. kaitai-io/kaitai_struct#457 is only some weird Q&A style discussion, I'd rather mark it with the blue label question. @GreyCat only mentions the creation of the https://github.com/kaitai-io/kaitai_compress repo, but there is no mention of any changes in KSC or runtime libraries. I think this is actually a made-up point: the The commit message states that the last change to that line was done when adding custom So please remove it:
Suggested change
|
||||||||||
* Deprecated classes and methods: | ||||||||||
* ~~`ensure_fixed_contents`~~ ⟶ explicit `if` that asserts `readBytes(n)` to be equal to the expected `n`-byte array (throwing `ValidationNotEqualError` if it fails) | ||||||||||
* ~~`UnexpectedDataError`~~ ⟶ `ValidationNotEqualError` | ||||||||||
|
@@ -99,12 +101,19 @@ fixes a lot of bugs and includes quite a few infrastructure improvements. | |||||||||
Gradle plugin](https://github.com/valery1707/kaitai-gradle-plugin) | ||||||||||
* Infrastructure updates: | ||||||||||
* Unstable binary builds are available for all platforms after every CI build at Bintray ([#63](https://github.com/kaitai-io/kaitai_struct/issues/63)) | ||||||||||
* KSY language reference replaced with [documentation](https://doc.kaitai.io/ksy_diagram.html) generated from JSON schema | ||||||||||
* KS language reference replaced with [documentation](https://doc.kaitai.io/ksy_diagram.html) generated from [the JSONSchema](https://github.com/kaitai-io/ksy_schema) | ||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It's called JSON Schema, not
Suggested change
|
||||||||||
* [https://formats.kaitai.io/](https://formats.kaitai.io/) is rebuilt automatically with CI/CD | ||||||||||
* Brand new modular CI/CD system for compiler, underlying | ||||||||||
CI-agnostic, working on multiple different OSes in parallel | ||||||||||
(Linux, Windows, macOS) and showing status at [https://ci.kaitai.io/](https://ci.kaitai.io/) | ||||||||||
* Generate test assertion specs from language-agnostic [KST specs](https://doc.kaitai.io/kst.html) | ||||||||||
* Related projects: | ||||||||||
* Added an [official library of compression algorithms](https://github.com/kaitai-io/kaitai_compress) to be used in `process`. | ||||||||||
* Created [a list of ![awesome](https://camo.githubusercontent.com/1997c7e760b163a61aba3a2c98f21be8c524be29/68747470733a2f2f617765736f6d652e72652f62616467652e737667) projects related to Kaitai Struct](https://github.com/kaitai-io/awesome-kaitai) | ||||||||||
Comment on lines
+111
to
+112
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Please keep the imperative style (add!, support!, allow!, implement!, ...) of the release notes.
Suggested change
|
||||||||||
* Alternative compiler implementations started: | ||||||||||
* [ksc-rs](https://github.com/Mingun/ksc-rs) for Rust | ||||||||||
* [kaitaigo](https://github.com/cugu/kaitaigo) for Go | ||||||||||
* [nimitai](https://github.com/sealmove/nimitai) for nim | ||||||||||
|
||||||||||
</div> | ||||||||||
|
||||||||||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would keep the word integers, because type is too broad in my opinion:
What was your intention when changing it to types?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Because endianness issue is not about bit-sized integers alone, it is about both bit- and byte-sized integers within types which fields (which can have non-integer type) can occupy non-integer count of bytes. Endeanness is here because of remapping. The common way to deal with bit-sized stuff is using bitwise logical operations, since it is the only mean of doing that on most of CPUs (maybe except some Intel ones with integrated FPGAs), and it is proficient to use CPU-native remapping, little-endian (no designers of modern systems are mad enough to use BE as native byte order). Such remapping usually deals with chunks longer than bytes, and it is position of numbers and structs relative to the chunks boundaries defines the way pieces of chunks are cut, reordered and joined. So, IMHO, endianness for bit-sized integers alone makes no sense as it is at all (how to cut
b9le
,0b11111111_1
, ,0b1_11111111
, or maybe0b11111_1111
?), what makes sense is the endianness of their container that has the bit-size of multiple of chunk length and aligned to chunks boundaries and where the field is within the container.