Create custom object type in Golang

[solganik]

Hi, i am trying to extend ast.Value interface to write my own custom object implementation.
the reason i am trying to do so is to achieve maximum performance without allocating external memory for my datastruct. I had unsuccessfully tried to implement my own type like this:

type MyCustomType struct {
	Id               ast.String 
	Name         ast.String 
}

func (p *MyCustomType) Compare(other ast.Value) int {
	if x, ok := other.(*MyCustomType); ok {
		return p.Id.Compare(x.Id)
	}
	return -1
}

func (p *MyCustomType) String() string {
	return p.Name.String()
}

func (p *MyCustomType) Find(path ast.Ref) (ast.Value, error) {
	if len(path) == 0 {
		return p, nil
	}

	if p0, ok := path[0].Value.(ast.String); ok {
		fieldName := string(p0)
		switch fieldName {
		case "Id":
			return p.Id, nil
		case "Name":
			return p.Name, nil
		}
	}
	return nil, fmt.Errorf("field %v not exists", path)
}

func (p *MyCustomType) IsGround() bool {
	return true
}

func (p *MyCustomType) Hash() int {
	return p.Id.Hash()
}

The reason i am doing it is because i already have a type that i manage internally and i want to perform policy evaluations on top of this type without allocating more memory or transforming it map and back.

However this samply policy evaluation fails,

allow := input.data.Name == "name"

it seems that the failure is originating from

func (e evalTerm) get(plugged *ast.Term) (*ast.Term, *bindings) {

only handling Object, Set and Array types. And it seems that Object cannot be externally extended.
Is there any way to create my own custom types in rego golang ?

[anderseknert]

I'm surprised it worked that far 😅 OPA isn't really buillt with user-extensible types in mind, from all I know.
There's plenty of places where we do e.g. switch statements on known Value implementations such, or in otherways assume the types are only those provided. I mean, you probably can make it work, but I would expect that to require much more surgery in the codebase than just implementing the Value interface.

There may be some other things to explore, like a custom ref resolver that would tap into your system on demand, or custom module loaders, etc... but at the end of the day, those would all have to resolve to something known to OPA. One benefit would be that it'd be an on-demand thing instead of having to convert all of your types upfront.

[solganik]

There is a lazy object type which resolves only accessed attributes, however this requires map of interfaces as an input (which in turn requires additional transformations and allocations)

[anderseknert]

Indeed, I don't think that's a way forward. The ref resolver is a different thing. As is the module loader. Whether any of them helps with your use case, I don't know... but they're at least options to explore.

[solganik]

Neither of them would work in my use-case, it seems though that Object interface "explicitly" prevents from creating another structs outside of "opa" package here:

ype Object interface {
	Value
	Len() int
	Get(*Term) *Term
	Copy() Object
	......
	get(k *Term) *objectElem // To prevent external implementations
}

Is there any reason why to prevent such thing for PDO (plain data object). this would significantly reduce CPU utilization when using SDK directly as this would allow to operate on top of "custom" structs without a need to json data back and forth.

I removed this constraint on my local dev env, implemented Get Method, and everything worked like this:

func (p * MyCustomType) Get(key *ast.Term) *ast.Term {
	if p0, ok := key.Value.(ast.String); ok {
		fieldName := string(p0)
		switch fieldName {
                case "Id":
                        return ast.NewTerm(p.Id)
		case "Name":
			return ast.NewTerm(p.Name)
		}
	}
	return nil
}

Can someone from Opa developers please comment as to why the restriction is in place and if that would be possible to ease this restriction to make much better utilization of CPU/Memory while using custom structs as sources of Input when using the SDK.

[anderseknert]

I can't comment on that, but I can see many potential issues with having to support custom implementations of basic types, so presumably related to that. If you could tell us a little more about your use case, perhaps we can find an alternative solution. Is it the input object you want to use a custom data type for?