Const schema refactoring #903

nielsenko · 2022-09-14T11:15:41Z

Const construct schema compile time

Dart has the ability to construct complex const objects compile time, which avoids any runtime overhead with regards to construction, and are never considered for garbage collection.

This is a very powerful feature, and it is used in Flutter to construct static widgets.

This PR explores the possibility of doing the same for our schema.

It introduces fly-weight property classes that are constructed compile time per realm model, and combined in a const constructed schema object.

These fly-weights are specialized by property type (ie. value, object, or list - set and map will come later), and help us deconstruct the static type of the collection, to get the static type of its elements.

This is useful to push type info and safety much deeper into the realm SDK, and allows us to avoid a fair amount runtime branching when reading from a realm. (The current PR doesn't do the same for writing yet)

It also refactor how we handle dynamic properties, which are now all accessed by get<T> including for lists, ie. get<List<T>>. The error messages are also improved.

The original idea came about, when implementing support for collections of nullable primitives, but was spun out into its own PR, as it is a significant change on its own.

The class hierarchy for realm objects changes slightly with this PR. It is now:

classDiagram

class RealmObject
<<abstract>> RealmObject
RealmObjectMarker <|.. RealmObject

class TypedRealmObject
<<abstract>> TypedRealmObject
RealmObject <|.. TypedRealmObject

class AGeneratedClass
RealmEntityMixin <|-- AGeneratedClass
RealmObjectMixin <|-- AGeneratedClass
TypedRealmObject <|.. AGeneratedClass

Here is an example of how the generated code looks like after this refactoring.

Given:

@RealmModel()
class _Player {
  @PrimaryKey()
  late String name;
  _Game? game;
  final scoresByRound = <int?>[]; // null means player didn't finish
}

@RealmModel()
class _Game {
  final winnerByRound = <_Player>[]; // I intended to support <_Player?> here as well, but alas core doesn't support it
  int get rounds => winnerByRound.length;
}

the generator produces:

class Player extends _Player
    with RealmEntityMixin, RealmObjectMixin<Player> {
  Player(
    String name, {
    Game? game,
    Iterable<int?> scoresByRound = const [],
  }) {
    _nameProperty.setValue(this, name);
    _gameProperty.setValue(this, game);
    _scoresByRoundProperty.setValue(this, RealmList<int?>(scoresByRound));
  }

  Player._();

  static const _nameProperty = ValueProperty<String>(
    'name',
    RealmPropertyType.string,
    primaryKey: true,
  );
  @override
  String get name => _nameProperty.getValue(this);
  @override
  set name(String value) => _nameProperty.setValue(this, value);

  static const _gameProperty = ObjectProperty<Game>('game', 'Game');
  @override
  Game? get game => _gameProperty.getValue(this);
  @override
  set game(covariant Game? value) => _gameProperty.setValue(this, value);

  static const _scoresByRoundProperty =
      ListProperty<int?>('scoresByRound', RealmPropertyType.int);
  @override
  RealmList<int?> get scoresByRound => _scoresByRoundProperty.getValue(this);
  @override
  set scoresByRound(covariant RealmList<int?> value) =>
      throw RealmUnsupportedSetError();

  static const schema = SchemaObject<Player>(
    Player._,
    'Player',
    {
      'name': _nameProperty,
      'game': _gameProperty,
      'scoresByRound': _scoresByRoundProperty,
    },
    _nameProperty,
  );
  @override
  SchemaObject get instanceSchema => schema;
}

class Game extends _Game
    with RealmEntityMixin, RealmObjectMixin<Game> {
  Game({
    Iterable<Player> winnerByRound = const [],
  }) {
    _winnerByRoundProperty.setValue(this, RealmList<Player>(winnerByRound));
  }

  Game._();

  static const _winnerByRoundProperty = ListProperty<Player>(
      'winnerByRound', RealmPropertyType.object,
      linkTarget: 'Player');
  @override
  RealmList<Player> get winnerByRound => _winnerByRoundProperty.getValue(this);
  @override
  set winnerByRound(covariant RealmList<Player> value) =>
      throw RealmUnsupportedSetError();

  static const schema = SchemaObject<Game>(
    Game._,
    'Game',
    {
      'winnerByRound': _winnerByRoundProperty,
    },
  );
  @override
  SchemaObject get instanceSchema => schema;
}

Notice how the schema is now static const, as it is assembled from the likewise static const property fly-weights.

nirinchev

Looks good, I have mostly clarification comments and minor suggestions

generator/lib/src/realm_field_info.dart

generator/lib/src/realm_object_generator.dart

lib/src/configuration.dart

lib/src/list.dart

lib/src/native/realm_core.dart

lib/src/realm_class.dart

nirinchev · 2022-09-21T11:43:35Z

lib/src/realm_class.dart

+    var object = RealmObjectInternal.create<T>(this, handle, metadata);
+    return object;


Suggested change

var object = RealmObjectInternal.create<T>(this, handle, metadata);

return object;

return RealmObjectInternal.create<T>(this, handle, metadata);

lib/src/realm_class.dart

lib/src/realm_object.dart

nirinchev · 2022-09-21T11:53:37Z

lib/src/realm_object.dart

-
-    return _values[name];
-  }
+  T getValue<T>(RealmObject object, String propertyName) => _values[propertyName] as T;


How does that work with default values? Like if I call getValue<int>(obj, age) and age doesn't exist in _values, wouldn't that throw?

Default values are handled at the property layer

Yes, but that's happening after calling this method. Here's an example of what I'm thinking about: https://dartpad.dev/?id=4d103b52e90ef328e32c12665a38a337

desistefanova

Added some questions?

desistefanova · 2022-09-21T06:48:37Z

common/lib/src/realm_types.dart

+const _intMapping = Mapping<int>();
+const _boolMapping = Mapping<bool>();
+const _doubleMapping = Mapping<double>();


Why this types are special and we have constants only for them?

because int, bool, and double are both members of the enum and primitive types in dart.. Hence this little trick

generator/lib/src/realm_object_generator.dart

test/realm_test.dart

desistefanova · 2022-09-21T14:02:46Z

I suppose even though this is a refactoring we will still have a line in the changelog about it.

…>>('x')

@PrimaryKey

Caveat to be aware of: ```dart @RealmModel() class _Person { @PrimaryKey() late String name; int age = 42; } final p = Person('Bob', age: 42); final p2 = Person('Bob')..age = 42; // behaves slightly differently wrt. sync ```

…ctor.

@nirinchev

…allows frozen and changes to be implemented without generator support, but doesn't introduce nearly as many changes. Based on feedback from @nirinchev

…sed map

desistefanova · 2022-10-12T16:52:15Z

generator/test/test_util.dart

+Future<dynamic> generatorTestBuilder(String directoryName, String inputFileName) async {
+  final inputPath = _path.join(directoryName, inputFileName);
+  final expectedPath = _path.setExtension(inputPath, '.expected');
+  return testBuilder(


This change breaks the idea that not all the input files could have a relevant expected result. It is possible the .expected file to be missing, because of some other logic for expecting exception or expecting no output. Why is it changed?

desistefanova · 2022-10-12T17:05:50Z

generator/test/test_util.dart

+      if (d.operation < 0) {
+        pen.red(bg: true); // delete
+      } else if (d.operation == 0) {
+        pen.reset(); // no-edit
+      } else if (d.operation > 0) {
+        pen.green(bg: true); // insert
+      }


Suggested change

if (d.operation < 0) {

pen.red(bg: true); // delete

} else if (d.operation == 0) {

pen.reset(); // no-edit

} else if (d.operation > 0) {

pen.green(bg: true); // insert

}

switch (d.operation) {

case DIFF_DELETE:

pen.red(bg: true);

break;

case DIFF_EQUAL:

pen.reset();

break;

case DIFF_INSERT:

pen.green(bg: true);

break;

default:

}

nirinchev

Another set of small suggestions

nirinchev · 2022-10-13T10:47:10Z

CHANGELOG.md

@@ -2,6 +2,9 @@

 **This project is in the Beta stage. The API should be quite stable, but occasional breaking changes may be made.**

+### Breaking Changes
+* Use const constructed schema objects. ([#903](https://github.com/realm/realm-dart/pull/903))


Is this a breaking change? I guess people just need to re-run the generator?

nirinchev · 2022-10-13T10:58:31Z

lib/src/native/realm_core.dart

+          final objectType = property.ref.link_target.cast<Utf8>().toRealmDartString(treatEmptyAsNull: true);
+          result &= schemaProperty.linkTarget == objectType;
+          final isNullable = property.ref.flags & realm_property_flags.RLM_PROPERTY_NULLABLE != 0;
+          result &= !(isNullable ^ schemaProperty.optional); // isNullable <=> schemaProperty.optional


That seems like an overly complicated way to express isNullable == schemaProperty.optional.

nirinchev · 2022-10-13T11:00:55Z

lib/src/native/realm_core.dart

@@ -2529,11 +2543,14 @@ extension on List<UserState> {
 extension on realm_property_info {
  SchemaProperty toSchemaProperty() {
    final linkTarget = link_target == nullptr ? null : link_target.cast<Utf8>().toDartString();


Suggested change

final linkTarget = link_target == nullptr ? null : link_target.cast<Utf8>().toDartString();

final linkTarget = link_target.cast<Utf8>().toRealmDartString(treatEmptyAsNull: true);

nirinchev · 2022-10-13T11:01:07Z

lib/src/native/realm_core.dart

+      propertyType: RealmPropertyType.values[type],
+      optional: flags & realm_property_flags.RLM_PROPERTY_NULLABLE == realm_property_flags.RLM_PROPERTY_NULLABLE,
+      primaryKey: flags & realm_property_flags.RLM_PROPERTY_PRIMARY_KEY == realm_property_flags.RLM_PROPERTY_PRIMARY_KEY,
+      linkTarget: linkTarget == null || linkTarget.isEmpty ? null : linkTarget,


Suggested change

linkTarget: linkTarget == null || linkTarget.isEmpty ? null : linkTarget,

linkTarget: linkTarget,

nielsenko · 2022-10-24T11:42:16Z

Postponed once more..

nielsenko requested a review from blagoev September 14, 2022 11:15

cla-bot bot added the cla: yes label Sep 14, 2022

nielsenko requested review from nirinchev, blagoev and desistefanova and removed request for blagoev and nirinchev September 14, 2022 11:15

nielsenko mentioned this pull request Sep 14, 2022

Allow nullable types in collections #708

Closed

nielsenko self-assigned this Sep 14, 2022

nielsenko changed the title ~~Kn/const schema refactoring~~ Const schema refactoring Sep 14, 2022

nielsenko force-pushed the kn/const-schema-refactoring branch 2 times, most recently from 45bd665 to 039bac7 Compare September 14, 2022 14:55

nielsenko force-pushed the kn/nullable-in-collections-no-refactoring branch 3 times, most recently from 958b123 to 20d975d Compare September 20, 2022 11:44

nirinchev reviewed Sep 21, 2022

View reviewed changes

nielsenko force-pushed the kn/nullable-in-collections-no-refactoring branch 3 times, most recently from 7a430df to f94c072 Compare September 21, 2022 13:25

desistefanova reviewed Sep 21, 2022

View reviewed changes

nielsenko force-pushed the kn/const-schema-refactoring branch 3 times, most recently from 9857287 to 36370c9 Compare September 23, 2022 08:56

nielsenko force-pushed the kn/nullable-in-collections-no-refactoring branch from f94c072 to b027b2c Compare September 23, 2022 11:40

nielsenko force-pushed the kn/const-schema-refactoring branch from 36370c9 to 92474cd Compare September 23, 2022 11:42

nielsenko force-pushed the kn/nullable-in-collections-no-refactoring branch 2 times, most recently from 043695c to a6ef9fc Compare September 23, 2022 20:00

nielsenko force-pushed the kn/const-schema-refactoring branch from 92474cd to 31a2a3c Compare September 23, 2022 20:41

nielsenko added 23 commits October 12, 2022 14:17

Get rid off DynamiRealmObject.getList<T>('x'). You can use get<List<T…

9b52515

…>>('x')

Decorate internal (but public) classes/enums with /// @Nodoc

0aefb15

Handle default values correctly against core

7ae42f4

Caveat to be aware of: ```dart @RealmModel() class _Person { @PrimaryKey() late String name; int age = 42; } final p = Person('Bob', age: 42); final p2 = Person('Bob')..age = 42; // behaves slightly differently wrt. sync ```

Add test case for default values (local realm only)

9238c80

Fix generator tests

c20df78

Make migration tests compile again

ddcf598

Make RealmObject interface generic on type

eb7a07b

Update generator to use generic RealmObjectMixin and RealmObject

b80d580

Run generator to update realm objects

f4ea3a0

Fix bug in create

625102a

Update .expected to match changes to generator

cba8206

Fix isInMigration bug

6e436f9

Tweak generator output a bit

ad31e7d

Remove bad comment from list.dart and cleanup a bit

165df3f

Avoid setting classInfo.primary_key twice

e000282

Strengthen assert to isNullable <=> schemaProperty.optional

fd4a878

Make objectFactory private on SchemaObject and allow to pass null in …

4bef442

…ctor.

Tweak default value test and add extra comment

0895676

Rebase changes

d74b40b

Split RealmObject<T> into RealmObject and TypedRealmObject<T>. Still …

d5f965d

…allows frozen and changes to be implemented without generator support, but doesn't introduce nearly as many changes. Based on feedback from @nirinchev

Fix all but 3 analyzer warnings

70769e6

Update CHANGELOG

d39de76

Address PR feedback about comments and lazy construction of string ba…

2bf0a3b

…sed map

nielsenko force-pushed the kn/const-schema-refactoring branch from 9931ebc to 2bf0a3b Compare October 12, 2022 12:17

desistefanova reviewed Oct 12, 2022

View reviewed changes

nirinchev reviewed Oct 13, 2022

View reviewed changes

nielsenko marked this pull request as draft October 24, 2022 11:41

nielsenko mentioned this pull request Dec 20, 2022

Realm Query export to CSV within the Flutter UI (Windows/macOS) #1076

Open

nielsenko mentioned this pull request Mar 12, 2024

RDART-910: Add support for RealmObject.objectSchema #1540

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Const schema refactoring #903

Const schema refactoring #903

nielsenko commented Sep 14, 2022 •

edited

Loading

nirinchev left a comment

nirinchev Sep 21, 2022

nirinchev Sep 21, 2022

nielsenko Sep 22, 2022

nirinchev Oct 13, 2022

desistefanova left a comment

desistefanova Sep 21, 2022 •

edited

Loading

nielsenko Sep 21, 2022

desistefanova commented Sep 21, 2022

desistefanova Oct 12, 2022

desistefanova Oct 12, 2022

nirinchev left a comment

nirinchev Oct 13, 2022

nirinchev Oct 13, 2022

nirinchev Oct 13, 2022

nirinchev Oct 13, 2022

nielsenko commented Oct 24, 2022

		var object = RealmObjectInternal.create<T>(this, handle, metadata);
		return object;

	var object = RealmObjectInternal.create<T>(this, handle, metadata);
	return object;
	return RealmObjectInternal.create<T>(this, handle, metadata);

	final linkTarget = link_target == nullptr ? null : link_target.cast<Utf8>().toDartString();
	final linkTarget = link_target.cast<Utf8>().toRealmDartString(treatEmptyAsNull: true);

	linkTarget: linkTarget == null \|\| linkTarget.isEmpty ? null : linkTarget,
	linkTarget: linkTarget,

Const schema refactoring #903

Are you sure you want to change the base?

Const schema refactoring #903

Conversation

nielsenko commented Sep 14, 2022 • edited Loading

nirinchev left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

desistefanova left a comment

Choose a reason for hiding this comment

desistefanova Sep 21, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

desistefanova commented Sep 21, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nirinchev left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nielsenko commented Oct 24, 2022

nielsenko commented Sep 14, 2022 •

edited

Loading

desistefanova Sep 21, 2022 •

edited

Loading