Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: remove State class #16352

Draft
wants to merge 69 commits into
base: develop
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
1971add
refactor: remove State class
thenswan Oct 31, 2024
91f73b7
fix: PTT unit tests
thenswan Oct 31, 2024
9a50283
Merge branch 'develop' into 15638-merkleroot--merklestateroot--state-…
thenswan Oct 31, 2024
c33c409
fix: add ConstructableIgnored annotations
thenswan Oct 31, 2024
84e2bbd
chore: update MTT and PTT
thenswan Oct 31, 2024
93719d9
Merge branch 'develop' into 15638-merkleroot--merklestateroot--state-…
thenswan Nov 4, 2024
cfbfa2d
update after merging develop
thenswan Nov 4, 2024
f28514d
Merge branch 'develop' into 15638-merkleroot--merklestateroot--state-…
thenswan Nov 4, 2024
aaaea3a
fix jrs
thenswan Nov 4, 2024
6df5bc8
spotless
thenswan Nov 4, 2024
c55bd10
fix turtle tests
thenswan Nov 4, 2024
282808e
fix init platform state
thenswan Nov 4, 2024
81f670a
Merge branch 'develop' into 15638-merkleroot--merklestateroot--state-…
thenswan Nov 4, 2024
67e63df
fix jrs
thenswan Nov 4, 2024
21a9891
add logging to NoOpMerkleStateLifecycles
thenswan Nov 4, 2024
4d2f136
ptt debugging
thenswan Nov 5, 2024
1f09146
ptt increase CHILD_COUNT
thenswan Nov 5, 2024
eb97518
Merge branch 'develop' into 15638-merkleroot--merklestateroot--state-…
thenswan Nov 5, 2024
a002085
ptt increase CHILD_COUNT
thenswan Nov 5, 2024
26692ab
spotless
thenswan Nov 5, 2024
c706c40
PTT CHILD_COUNT experiment
thenswan Nov 5, 2024
5a59b72
PTT debugging
thenswan Nov 5, 2024
b27ad3b
PTT fixing
thenswan Nov 5, 2024
579517a
PTT fixing
thenswan Nov 5, 2024
22c7909
PTT fixing
thenswan Nov 5, 2024
15b0c32
fix: PTT
thenswan Nov 7, 2024
85a5cd0
Merge branch 'develop' into 15638-merkleroot--merklestateroot--state-…
thenswan Nov 7, 2024
6db8463
fix: compilation error
thenswan Nov 7, 2024
9dcbc14
ptt debugging
thenswan Nov 7, 2024
2caa5a3
ptt debugging
thenswan Nov 7, 2024
a8f26b5
ptt debugging
thenswan Nov 7, 2024
6d6216a
ptt fixing
thenswan Nov 7, 2024
739900d
ptt debugging
thenswan Nov 8, 2024
313a04b
ptt debugging
thenswan Nov 8, 2024
1f0e472
ptt debugging
thenswan Nov 8, 2024
1467726
ptt debugging
thenswan Nov 8, 2024
13f1b54
ptt debugging
thenswan Nov 8, 2024
e1baae0
ptt debugging
thenswan Nov 8, 2024
700548c
commented logging
thenswan Nov 11, 2024
d7ccbf5
Merge branch 'develop' into 15638-merkleroot--merklestateroot--state-…
thenswan Nov 11, 2024
3a4e627
spotless
thenswan Nov 11, 2024
9dfba2d
ptt fix
thenswan Nov 11, 2024
bdad0c2
Added more logging
imalygin Nov 11, 2024
d548537
Added registration for ConsistencyTestingToolState
imalygin Nov 12, 2024
bf9582c
ptt debugging
thenswan Nov 12, 2024
d1aa4cb
Moved PlatformTestingToolState registration to PlatformTestingToolMai…
imalygin Nov 12, 2024
829a91f
Spotless
imalygin Nov 12, 2024
04819db
Moved ConsistencyTestingToolState registration to ConsistencyTestingT…
imalygin Nov 12, 2024
42fc9cc
Spotless
imalygin Nov 12, 2024
8855f57
Fixed minimum supported version.
imalygin Nov 12, 2024
164682b
Added logging of registered classId-s
imalygin Nov 12, 2024
ecddfe9
Merge branch 'develop' into 15638-merkleroot--merklestateroot--state-…
imalygin Nov 12, 2024
60f973c
Merge branch 'develop' into 15638-merkleroot--merklestateroot--state-…
imalygin Nov 12, 2024
26282e8
Spotless
imalygin Nov 12, 2024
9016561
Added missing dependency
imalygin Nov 12, 2024
e70985b
Added registration of the PlatformState classes
imalygin Nov 12, 2024
7f3b4ad
Added registration of the PlatformState classes
imalygin Nov 13, 2024
751c5bb
Added missing dependency
imalygin Nov 13, 2024
fa15fb6
Another attempt to register platform classes.
imalygin Nov 13, 2024
91703de
Added more logging
imalygin Nov 13, 2024
022ce6b
Merge branch 'develop' into 15638-merkleroot--merklestateroot--state-…
imalygin Nov 13, 2024
a53a4a0
Added leafs for `stateLong` and `roundsHandled` and utilized them
imalygin Nov 14, 2024
94f60f1
Spotless
imalygin Nov 15, 2024
2977d19
fix initialization issue
thenswan Nov 15, 2024
991dad4
fix initialization issue
thenswan Nov 15, 2024
207d964
fix initialization issue
thenswan Nov 15, 2024
c31e88e
fix ConsistencyTestingToolState init
thenswan Nov 15, 2024
6ed59af
Moved service class registration to `StateUtils` of `swirlds-state-im…
imalygin Nov 15, 2024
fb3011b
Fixed dependencies
imalygin Nov 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
import static com.hedera.node.app.state.merkle.VersionUtils.alreadyIncludesStateDefs;
import static com.hedera.node.app.state.merkle.VersionUtils.isSoOrdered;
import static com.hedera.node.app.workflows.handle.metric.UnavailableMetrics.UNAVAILABLE_METRICS;
import static com.swirlds.state.merkle.StateUtils.registerWithSystem;
import static java.util.Objects.requireNonNull;

import com.hedera.hapi.node.base.SemanticVersion;
Expand All @@ -31,9 +32,7 @@
import com.hedera.node.app.services.MigrationStateChanges;
import com.hedera.node.app.spi.state.FilteredReadableStates;
import com.hedera.node.app.spi.state.FilteredWritableStates;
import com.swirlds.common.constructable.ClassConstructorPair;
import com.swirlds.common.constructable.ConstructableRegistry;
import com.swirlds.common.constructable.ConstructableRegistryException;
import com.swirlds.common.crypto.DigestType;
import com.swirlds.config.api.Configuration;
import com.swirlds.merkle.map.MerkleMap;
Expand All @@ -51,16 +50,10 @@
import com.swirlds.state.lifecycle.info.NetworkInfo;
import com.swirlds.state.merkle.StateMetadata;
import com.swirlds.state.merkle.StateUtils;
import com.swirlds.state.merkle.disk.OnDiskKey;
import com.swirlds.state.merkle.disk.OnDiskKeySerializer;
import com.swirlds.state.merkle.disk.OnDiskValue;
import com.swirlds.state.merkle.disk.OnDiskValueSerializer;
import com.swirlds.state.merkle.memory.InMemoryValue;
import com.swirlds.state.merkle.memory.InMemoryWritableKVState;
import com.swirlds.state.merkle.queue.QueueNode;
import com.swirlds.state.merkle.singleton.SingletonNode;
import com.swirlds.state.merkle.singleton.StringLeaf;
import com.swirlds.state.merkle.singleton.ValueLeaf;
import com.swirlds.state.spi.WritableStates;
import com.swirlds.virtualmap.VirtualMap;
import edu.umd.cs.findbugs.annotations.NonNull;
Expand Down Expand Up @@ -151,7 +144,7 @@ public SchemaRegistry register(@NonNull Schema schema) {
schema.statesToCreate(bootstrapConfig).forEach(def -> {
//noinspection rawtypes,unchecked
final var md = new StateMetadata<>(serviceName, schema, def);
registerWithSystem(md);
registerWithSystem(md, constructableRegistry);
});

return this;
Expand Down Expand Up @@ -380,81 +373,4 @@ private RedefinedWritableStates applyStateDefinitions(
final var newStates = new FilteredWritableStates(writableStates, remainingStates);
return new RedefinedWritableStates(writableStates, newStates);
}

/**
* Registers with the {@link ConstructableRegistry} system a class ID and a class. While this
* will only be used for in-memory states, it is safe to register for on-disk ones as well.
*
* <p>The implementation will take the service name and the state key and compute a hash for it.
* It will then convert the hash to a long, and use that as the class ID. It will then register
* an {@link InMemoryWritableKVState}'s value merkle type to be deserialized, answering with the
* generated class ID.
*
* @param md The state metadata
*/
@SuppressWarnings({"rawtypes", "unchecked"})
private void registerWithSystem(@NonNull final StateMetadata md) {
// Register with the system the uniqueId as the "classId" of an InMemoryValue. There can be
// multiple id's associated with InMemoryValue. The secret is that the supplier captures the
// various delegate writers and parsers, and so can parse/write different types of data
// based on the id.
try {
constructableRegistry.registerConstructable(new ClassConstructorPair(
InMemoryValue.class,
() -> new InMemoryValue(
md.inMemoryValueClassId(),
md.stateDefinition().keyCodec(),
md.stateDefinition().valueCodec())));
constructableRegistry.registerConstructable(new ClassConstructorPair(
OnDiskKey.class,
() -> new OnDiskKey<>(
md.onDiskKeyClassId(), md.stateDefinition().keyCodec())));
constructableRegistry.registerConstructable(new ClassConstructorPair(
OnDiskKeySerializer.class,
() -> new OnDiskKeySerializer<>(
md.onDiskKeySerializerClassId(),
md.onDiskKeyClassId(),
md.stateDefinition().keyCodec())));
constructableRegistry.registerConstructable(new ClassConstructorPair(
OnDiskValue.class,
() -> new OnDiskValue<>(
md.onDiskValueClassId(), md.stateDefinition().valueCodec())));
constructableRegistry.registerConstructable(new ClassConstructorPair(
OnDiskValueSerializer.class,
() -> new OnDiskValueSerializer<>(
md.onDiskValueSerializerClassId(),
md.onDiskValueClassId(),
md.stateDefinition().valueCodec())));
constructableRegistry.registerConstructable(new ClassConstructorPair(
SingletonNode.class,
() -> new SingletonNode<>(
md.serviceName(),
md.stateDefinition().stateKey(),
md.singletonClassId(),
md.stateDefinition().valueCodec(),
null)));
constructableRegistry.registerConstructable(new ClassConstructorPair(
QueueNode.class,
() -> new QueueNode<>(
md.serviceName(),
md.stateDefinition().stateKey(),
md.queueNodeClassId(),
md.singletonClassId(),
md.stateDefinition().valueCodec())));
constructableRegistry.registerConstructable(new ClassConstructorPair(StringLeaf.class, StringLeaf::new));
constructableRegistry.registerConstructable(new ClassConstructorPair(
ValueLeaf.class,
() -> new ValueLeaf<>(
md.singletonClassId(), md.stateDefinition().valueCodec())));
} catch (ConstructableRegistryException e) {
// This is a fatal error.
throw new IllegalStateException(
"Failed to register with the system '"
+ serviceName
+ ":"
+ md.stateDefinition().stateKey()
+ "'",
e);
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,11 @@
import com.swirlds.common.utility.AutoCloseableWrapper;
import com.swirlds.metrics.api.Metrics;
import com.swirlds.platform.Browser;
import com.swirlds.platform.state.MerkleRoot;
import com.swirlds.platform.state.State;
import com.swirlds.platform.state.MerkleStateRoot;
import com.swirlds.platform.system.BasicSoftwareVersion;
import com.swirlds.platform.system.Platform;
import com.swirlds.platform.system.SwirldMain;
import com.swirlds.platform.util.NoOpMerkleStateLifecycles;
import edu.umd.cs.findbugs.annotations.NonNull;
import java.awt.event.KeyEvent;
import java.awt.event.KeyListener;
Expand Down Expand Up @@ -182,9 +182,11 @@ public void run() {

@Override
@NonNull
public MerkleRoot newMerkleStateRoot() {
final State state = new State();
state.setSwirldState(new CryptocurrencyDemoState());
public MerkleStateRoot newMerkleStateRoot() {
final MerkleStateRoot state = new CryptocurrencyDemoState(
NoOpMerkleStateLifecycles.NO_OP_MERKLE_STATE_LIFECYCLES,
version -> new BasicSoftwareVersion(softwareVersion.getSoftwareVersion()));
NoOpMerkleStateLifecycles.NO_OP_MERKLE_STATE_LIFECYCLES.initPlatformState(state);
return state;
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,32 +26,31 @@
* DISTRIBUTING THIS SOFTWARE OR ITS DERIVATIVES.
*/

import com.hedera.hapi.node.base.SemanticVersion;
import com.hedera.pbj.runtime.io.buffer.Bytes;
import com.swirlds.common.io.streams.SerializableDataInputStream;
import com.swirlds.common.io.streams.SerializableDataOutputStream;
import com.swirlds.common.merkle.MerkleLeaf;
import com.swirlds.common.merkle.impl.PartialMerkleLeaf;
import com.swirlds.common.constructable.ConstructableIgnored;
import com.swirlds.common.platform.NodeId;
import com.swirlds.platform.SwirldsPlatform;
import com.swirlds.platform.state.MerkleStateLifecycles;
import com.swirlds.platform.state.MerkleStateRoot;
import com.swirlds.platform.state.PlatformStateModifier;
import com.swirlds.platform.system.InitTrigger;
import com.swirlds.platform.system.Platform;
import com.swirlds.platform.system.Round;
import com.swirlds.platform.system.SoftwareVersion;
import com.swirlds.platform.system.SwirldState;
import com.swirlds.platform.system.address.Address;
import com.swirlds.platform.system.address.AddressBook;
import com.swirlds.platform.system.transaction.Transaction;
import edu.umd.cs.findbugs.annotations.NonNull;
import edu.umd.cs.findbugs.annotations.Nullable;
import java.io.IOException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Random;
import java.util.concurrent.atomic.AtomicLong;
import java.util.function.Function;

/**
* This holds the current state of a swirld representing both a cryptocurrency and a stock market.
Expand All @@ -62,7 +61,8 @@
* entirely new cryptocurrency is created every time all the computers start the program over again, so
* these cryptocurrencies won't have any actual value.
*/
public class CryptocurrencyDemoState extends PartialMerkleLeaf implements SwirldState, MerkleLeaf {
@ConstructableIgnored
public class CryptocurrencyDemoState extends MerkleStateRoot {

/**
* The version history of this class.
Expand Down Expand Up @@ -134,7 +134,11 @@ public static enum TransType {

////////////////////////////////////////////////////

public CryptocurrencyDemoState() {}
public CryptocurrencyDemoState(
@NonNull final MerkleStateLifecycles lifecycles,
@NonNull final Function<SemanticVersion, SoftwareVersion> versionFactory) {
super(lifecycles, versionFactory);
}

private CryptocurrencyDemoState(final CryptocurrencyDemoState sourceState) {
super(sourceState);
Expand Down Expand Up @@ -196,6 +200,7 @@ public synchronized long getNumTrades() {
@Override
public synchronized CryptocurrencyDemoState copy() {
throwIfImmutable();
setImmutable(true);
return new CryptocurrencyDemoState(this);
}

Expand Down Expand Up @@ -380,76 +385,14 @@ public void init(
@NonNull final Platform platform,
@NonNull final InitTrigger trigger,
@Nullable final SoftwareVersion previousSoftwareVersion) {
this.platform = (SwirldsPlatform) platform;
super.init(platform, trigger, previousSoftwareVersion);

this.platform = (SwirldsPlatform) platform;
if (trigger == InitTrigger.GENESIS) {
genesisInit();
}
}

@Override
public void serialize(final SerializableDataOutputStream out) throws IOException {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure how important this particular state implementation is, but this serialization/deserialization logic cannot be just removed. However, this class may be no longer relevant, and it doesn't make sense spend too much time fixing it

out.writeStringArray(tickerSymbol);

out.writeInt(wallet.size());
for (Map.Entry<NodeId, AtomicLong> entry : wallet.entrySet()) {
out.writeSerializable(entry.getKey(), false);
out.writeLong(entry.getValue().get());
}

out.writeInt(shares.size());
for (Map.Entry<NodeId, List<AtomicLong>> entry : shares.entrySet()) {
out.writeSerializable(entry.getKey(), false);
out.writeInt(entry.getValue().size());
for (int i = 0; i < entry.getValue().size(); i++) {
out.writeLong(entry.getValue().get(i).get());
}
}

out.writeStringArray(trades);
out.writeInt(numTradesStored);
out.writeInt(lastTradeIndex);
out.writeLong(numTrades);
out.writeByteArray(ask);
out.writeByteArray(bid);
out.writeSerializableArray(askId, false, true);
out.writeSerializableArray(bidId, false, true);
out.writeByteArray(price);
}

@Override
public void deserialize(final SerializableDataInputStream in, final int version) throws IOException {
tickerSymbol = in.readStringArray(DEFAULT_MAX_ARRAY_SIZE, DEFAULT_MAX_STRING_SIZE);

int walletSize = in.readInt();
for (int i = 0; i < walletSize; i++) {
final NodeId id = in.readSerializable(false, NodeId::new);
final AtomicLong amount = new AtomicLong(in.readLong());
wallet.put(id, amount);
}

final int sharesSize = in.readInt();
for (int i = 0; i < sharesSize; i++) {
final NodeId id = in.readSerializable(false, NodeId::new);
final int numShares = in.readInt();
final List<AtomicLong> sharesForID = new ArrayList<>(numShares);
for (int j = 0; j < numShares; j++) {
sharesForID.add(new AtomicLong(in.readLong()));
}
shares.put(id, sharesForID);
}

trades = in.readStringArray(DEFAULT_MAX_ARRAY_SIZE, DEFAULT_MAX_STRING_SIZE);
numTradesStored = in.readInt();
lastTradeIndex = in.readInt();
numTrades = in.readLong();
ask = in.readByteArray(DEFAULT_MAX_ARRAY_SIZE);
bid = in.readByteArray(DEFAULT_MAX_ARRAY_SIZE);
askId = in.readSerializableArray(NodeId[]::new, DEFAULT_MAX_ARRAY_SIZE, false, NodeId::new);
bidId = in.readSerializableArray(NodeId[]::new, DEFAULT_MAX_ARRAY_SIZE, false, NodeId::new);
price = in.readByteArray(DEFAULT_MAX_ARRAY_SIZE);
}

@Override
public long getClassId() {
return CLASS_ID;
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
module com.swirlds.demo.crypto {
requires com.hedera.node.hapi;
requires com.swirlds.common;
requires com.swirlds.metrics.api;
requires com.swirlds.platform.core;
requires com.swirlds.state.api;
requires com.hedera.pbj.runtime;
requires java.desktop;
requires static com.github.spotbugs.annotations;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,12 +35,12 @@
import com.swirlds.platform.SwirldsPlatform;
import com.swirlds.platform.listeners.PlatformStatusChangeListener;
import com.swirlds.platform.listeners.PlatformStatusChangeNotification;
import com.swirlds.platform.state.MerkleRoot;
import com.swirlds.platform.state.State;
import com.swirlds.platform.state.MerkleStateRoot;
import com.swirlds.platform.system.BasicSoftwareVersion;
import com.swirlds.platform.system.Platform;
import com.swirlds.platform.system.SwirldMain;
import com.swirlds.platform.system.status.PlatformStatus;
import com.swirlds.platform.util.NoOpMerkleStateLifecycles;
import edu.umd.cs.findbugs.annotations.NonNull;
import java.nio.charset.StandardCharsets;

Expand Down Expand Up @@ -109,9 +109,11 @@ public void run() {

@NonNull
@Override
public MerkleRoot newMerkleStateRoot() {
final State state = new State();
state.setSwirldState(new HelloSwirldDemoState());
public MerkleStateRoot newMerkleStateRoot() {
final MerkleStateRoot state = new HelloSwirldDemoState(
NoOpMerkleStateLifecycles.NO_OP_MERKLE_STATE_LIFECYCLES,
version -> new BasicSoftwareVersion(softwareVersion.getSoftwareVersion()));
NoOpMerkleStateLifecycles.NO_OP_MERKLE_STATE_LIFECYCLES.initPlatformState(state);
return state;
}

Expand Down
Loading
Loading