-
Notifications
You must be signed in to change notification settings - Fork 4
Work Session Summaries Archive 2018Q4 2019Q1
Agenda:
- Align the "Task API" vs "Activity API scope" with PR and MF.
Outcomes
- We have reviewed the scopes of "Activity API" and its capability levels/segments:
- Control (start, abort, suspend, resume)
- Transport ("input", "output", streaming)
- State (GetState, GetUsage)
- Events (...ability to define, publish and subscribe to events raised by app hosted Provider...)
- Persist (dump, restore)
- We have identified a concept of "ExeUnit" - a component instantiated by the Golem Provider daemon to run Activities on. The ExeUnit "hosts" the payload application, so that it runnable and controllable within Golem ecosystem. The ExeUnit needs an interface specification, and there needs to be correspondence between the Activity API segments and the ExeUnit interface segments.
- Marek with Clay team have begun work on "Simple Task API" which specifies the interaction between the application hosted by a Golem daemon and the ExeUnit it is hosted in.
- We propose to progress work on Activity API, by following sequence:
- pick a capability segment, draft its scope and operations.
- define corresponding operations in ExeUnit interface.
- review impact of ExeUnit interface on the "Task API" concepts - and align/resolve discrepancies.
Actions:
- Establish the BB05 Golem Activity Protocol building block document.
- Record the Activity API Capability segments.
- Review and agree the Activity state diagram.
- Pick the first capability segment and propose draft API operations as well as ExeUnit operations.
- Invite Marek F. to "even" Arch workshops for alignment
Agenda:
- Share and review the current state of Golem Standards stub - agree next steps.
- Backlog review
Agenda:
- Graphene ng vision sharing
Agenda:
- Return to Provider Metamodel and review the activity flow in Provider implementation (relate to the Market API DIstributed Protocol specs content)
- Need to start outlining Activity API and Payment API.
- Status update ad Market API Distributed Protocol Implementation specs
Homework:
- PR to do feasibility study of Market API to Brass Golem tunneling (poc implementation of Market API on top of Brass Golem protocol)
Agenda:
- Review of Baltic LSC workshop
- Update to R&D team on the progress on Golem Computing Resource Standards
Homework:
- MB: Adjust the D&O Language Specs to latest syntax (props and samples). Also replace all .time.sec with duration_sec.
- MB: Invent and unify the definition of usage vector property (golem.svc.usagevector vs golem.usage.vector???)
- MB: Refactor the Provider model diagram to reflect the revised discussed design
Agenda:
-
Review Golem Standards breakdown structure - check homework, propose assignment of standard definition work to people?
- Expected product: proposed task plan of distributing work on Standards.
-
[by P.Chr] naming clash: Marketplace vs Market API
it appears (at least to MP) that names: Marketplace (R&D is working on) and Market API (our work) can be confusing in communication. Do we need to rename one of them or just describe carefully?
We have covered diverse TEE (mostly SGX) subjects and possible use-cases:
- SGX in gaming dev (supporting parts of game servers in TEE)
- bridge between game dev and BC
- server logic balancing (part of this logic can be hosted on clinets' machines)
- fully decentralized game servers
- DRM
- atomic swap
- DEX
- license enforcing/checking via SGX (e.g., local trusted license servers)
- docker under Win via graphene
- decentralized services
- e.g., streaming service (client may also be required to have some sort of TEE enabled)
- e.g., already mentioned game servers
- e.g., computational chemistry service (data confidentiality as one of the most important concerns)
We also considered what to do next within Golem Unlimited
- WASM: this fruit is hanging much lower, but it seems less desirable from user perspective
- SGX: technology is not yet mature and usable, but it could attract more users
- distributed compilation in TEE (studios do it all the time, utilizing resources from data centers; looks like a good fit for Unlimited)
- cooking in TEE
- parts of pipelines in TEE
Agenda:
- Review Golem Standards breakdown structure - check homework, added new standard areas & concepts (eg. caps, net, examples)
Agenda:
-
Define Golem Standards breakdown structure - define areas and hierarchy so that work on standards can be scaled across the team.
-
Return to Provider Metamodel and review the activity flow in Provider implementation
-
Status update ad Market API Distributed Protocol Implementation specs
-
[by P.Chr] naming clash: Marketplace vs Market API
it appears (at least to MP) that names: Marketplace (R&D is working on) and Market API (our work) can be confusing in communication. Do we need to rename one of them or just describe carefully?
-
Decide the next steps in "Golem licenses" thread?
- it seems there needs to be a decision of principle as to what level of legal compliance is expected.
Agenda:
- Recap the Golem Standard stub structure
Outcomes
- (Licenses) R&D squad needs to regroup and reconsider the licensing model. When ready - the adjusted vision shall be raised to "Arch workshop" forum with intent of placing license-related interactions within the *API context. Signal from R&D is expected when they are ready to return to the subject.
Agenda:
- Licensing model chat - first review of comments to document
Outcomes
-
Licensing is a complex and multidimensional problem and requires further decomposition to isolate scenarios/cases to focus on.
-
One of scenario categories is license service implemented on blockchain - this looks like a separate (interesting) project where coupling with mainstream Golem implementation seems marginal (ie. Golem Apps may want to use such service, but it is not a dependency for Golem implementation, nor existing APIs)
- It would be worthwhile to include (in the license article) a study of existing blockchain-base software licensing solutions.
-
A question of App Manifest and its mandatory status:
- The main benefit of such manifest is a binding of software package to licensing model & information.
- A separate App Manifest seems redundant, as:
- A Golem App (specific "integration": use case implementation UI) needs to be downloaded to Requestor node - and it includes logic to formulate Demands
- Demands include app description properties and relevant licensing properties (ie. the binding of license is implied by the Golem App itself)
- According to licensing properties the Provider side (ie. it's execution environment) should be able to validate the license (ie. calculate thumbprint of relevant payload to validate against License Service, etc.) - but the specifics of this activity are still very vague
- An App Store (as a catalogue of App Manifests) seems useful, but again - not really coupled with Golem Network itself. Ie. Golem Ecosystem does not require App Store to function properly.
-
Casual question - has MS Windows license conditions been validated - ie. do they allow "selling" computing power in a network such as Golem?
Homework
- ?? Need to specify the sequence of Simple Golem License usage scenario (the simplest case) - to identify any interactions/impact on Market API/Activity API.
- Eg. Is Simple Golem License resolution a case for property aspects maybe???
- LG/WN: Liaise with Hoard team to exchange thoughts on TaaL concept.
- It seems we need a follow-up. This time with objectives:
- Narrow-down the scope of focus.
- Formulate next steps
Agenda:
- Discuss the multi-activity scenario of "dynamic sub tasks in video transcoding" in the context of Market API (raised by Radek)
Agenda:
- Review of standard artifact stubs in repo
- Re "Standardization Framework" - What are sensible dimensions which should be standardized?
- Return to Provider Metamodel and review the activity flow in Provider implementation
- Status update ad Implementation specs
Homework:
- PCh: Adjust standard stubs to layout agreed on meeting.
- MB: Adjust Resolver poc to filter criteria format (types in constraints)
Agenda:
- Meet MP to introduce to Golem Arch Roadmap
- Run through the Arch Method slidedeck
- Show current Building Blocks and their artifacts
- Show the subsequent actions spreadsheet and discuss Trello
Homework:
- MP: Explore the architectural document repository starting from "D&O Matching - Problem Statement". Provide feedback on what content seems potentially useful for external communication of Golem Arch proceedings.
Agenda:
- Consider content that could be used for external publication - Invited MP for a walkthrough session on Tuesday
- Started standards documentation stub in golem-architecture repository (drafts branch)
Homework:
- PCh: Walkthrough https://docs.google.com/document/d/1tzMrhdBr9wiUXtSn1JO18MmIiP31dkMakdjStnF3eZY/edit# and move the property samples into relevant property categories in https://github.com/golemfactory/golem-architecture/tree/draft/standards
Backlog:
- Homework:
- All: Re "Standardization Framework" - What are sensible dimensions which should be standardized?
Agenda:
- Status update ad Implementation specs
- Apply updates to D&O Specification Language
- High-level walk through the remaining APIs/modules and sketch their interfaces, for the purpose of drawing a high-level end-2-end control flow through Golem software.
Homework:
- MB: Prepare summary of topics to be covered and propose sequence & approach.
Backlog:
- Homework:
- All: Re "Standardization Framework" - What are sensible dimensions which should be standardized?
Agenda:
- Stamp the Market API description as (stable)
- Build a compelling Brass Blender (PR to extract sensible property examples for a valid description)
- Discuss implementation of aspects
- Status update ad Implementation specs
- High-level walk through the remaining APIs/modules and sketch their interfaces, for the purpose of drawing a high-level end-2-end control flow through Golem software.
Backlog:
- Homework:
- All: Re "Standardization Framework" - What are sensible dimensions which should be standardized?
Output
- MB: Send invite for Tuesday to: team leads.
- PR&MB hone the input to Brass Blender example.
Agenda:
- Stamp the Market API description as (stable)
- Discuss implementation of aspects
- Status update ad Implementation specs
- High-level walk through the remaining APIs/modules and sketch their interfaces, for the purpose of drawing a high-level end-2-end control flow through Golem software.
Backlog:
- Homework:
- All: Re "Standardization Framework" - What are sensible dimensions which should be standardized?
Output
- Do we still need another "Wait" to listen for TerminateAgreement events, or does this belong to Activity API?
- Is TerminateAgreement a common operation?
Agenda:
- Stamp the Market API description as (stable) --- yeah, you wish... :P
Output
- Do we still need another "Wait" to listen for TerminateAgreement events, or does this belong to Activity API?
- Is TerminateAgreement a common operation?
Agenda:
- Discuss with R&D (Lukasz Glen) the topics they had raised.
- Reiterate the shape of Market API, verify methods, parameters and results (as per Golem API Spec Section 5.2.2. and Market API Swagger specs in https://github.com/golemfactory/golem-model/tree/master/market/spec)
Backlog:
- High-level walk through the remaining APIs/modules and sketch their interfaces, for the purpose of drawing a high-level end-2-end control flow through Golem software.
- Homework:
- All: Re "Standardization Framework" - What are sensible dimensions which should be standardized?
Time used for general catch-up on Golem situation and technical chat on low-level aspects of Market API.
Homework (for the week):
- (MB) Fill the gaps in Market API documentation (eg. remaining implementation options, put descriptions in to have it over with, etc.)
- (MB) Read and understand the "Anti-sybill Model 0"
- (MB) Continue on Demand&Offer Resolver prototype
- (PR&PCh&AS?) Build draft specs of Market API Dedicated Protocol specs (GitHub markdown file in https://github.com/golemfactory/golem-model/tree/master/market/spec?):
- Overview of Requestor & Provider side's implementation elements
- Description of how subscription works on each side (free-form diagram? UML communication diagram? UML sequence diagram?)
- Draft layout of messages used in the protocol
- For each of Market API methods - behaviour summary and/or pseudocode
Agenda:
- Follow-up of Market API review, discussion on method signatures and semantics:
- Collect() - questions about objects returned by collect() call - are we ok with collect() returning both Demands/Offers and Dynamic Property resolution calls?
- Post() - if Post() is to be used for Demand/Offer negotiation, should it include the "accept"/"reject" flag?
Homework:
- (MB) Rewrite the Market API functions into "functions" (see comments in API doc) and integrate comments.
- eg. collect() on Requestor side must return both 'public' and 'private' Offers
- (MB) Need to restore the find/scan() function (on both sides) - what would be the parameter of such function??? Offer/Demand?
- (All) For consideration:
- Alt-Golem compatibility - can Alt-Golem be implemented on MarketAPI, assuming that:
- Alt-Golem matching mechanism is based on a Market server (trusted)
- Alt-Golem is Data centre focused
- Alt-Golem handles "bulk offers"
- Unlimited
- Can Unlimited handle "bulk offers"?
- Do we want to distinguish "Suspended" agreement state???
Backlog:
- High-level walk through the remaining APIs/modules and sketch their interfaces, for the purpose of drawing a high-level end-2-end control flow through Golem software.
- Homework:
- All: Re "Standardization Framework" - What are sensible dimensions which should be standardized?
Agenda:
- Reiteration of the interfaces of Market API and logic behind the methods. Market API defined as split in two sides, Requestor and Provider, each with following methods:
- Subscribe()
- Collect()
- Post()
Agenda:
- Review & reiterate the Market API operations? Is it sufficient, generic enough?
- Use Przemek's example first (where the requested service parameters change between "discovery" and "contract signing")
- Review homework:
- All: Do we allow Offers without pricing function (for "open" Offers)?
Homework:
- MB: integrate comments in the Language and Matching documents
- MB: send out the links to Contract API/Agreement handshake
Agenda:
- Adjust timing for regular catch-ups
- Review diagram from JR?
Homework:
- MB: reach out to dev team for support in designing specs for Market API Implementation (done)
- MB: Integrate all pending comments in docs
- MB: Provide comments to the JR's high-level diagram
Notes:
- Updated "Demand&Offer Specification Language" to add parametric/dynamic property syntax.
Agenda:
- Review the proposed Demand&Offer syntax changes
- Adjust timing for regular catch-ups
Participants: PJ, PR, PCh, MBenke, MB
Agenda:
- Discuss options for more frequent interaction (daily catch-ups?)
- Review Hybrid Implementation
- Including the 2 scenarios description given as homework
- Review draft Matching Use Case scenarios to cover.
** Notes **
- For R&D squad for consideration - should we also consider a scenario of realistic interactive (presumably: pay-as-you-go) service? Progressive rendering? Transcoding?
** Homework **
- All: Do we allow Offers without pricing function (for "open" Offers)?
- All: Re "Standardization Framework" - What are sensible dimensions which should be standardized?
- PR*PCh: Update the Hybrid Implementation description with specs for network nodes' responsibilities, including the mechanism for fetching "dynamic" properties.
- MB: Think and porpose alternative syntax for prametrized properties.
- MB: Send invites for daily catchups (12-13)
Participants: PJ, PR, PCh, MB
Agenda:
- Walk through the proposed Service Standardization rules.
- Walk through the amendments to Demand&Offer Specs Language changes (Strong/Weak Matching, Strength Metrics)
- Start building draft Matching Use Case scenarios to cover.
- Walk through the Hybrid Implementation concept ideas.
Homework:
- MB: Rewrite the Golem Brass Task protocol into the Matching doc.
- MB: Elaborate on Service Standards structures (examples needed for both hierarchical and multidimensional) - Question: how can we define the decomposition rules for categories/dimension? In other words: what approach do we take to define the standard?
- MB: Elaborate more on anti-spam requirements in Market API as well as Contract API (to prevent spamming there must be some cost incurred on sending party)
- PR/PCh: Please elaborate on implementation of two scenarios in Hybrid Matching mechanism?
- MB: Propose options for more frequent iterations/catch-ups.
Participants: PJ, PR, PCh, MB
Agenda:
- Review the restructuring of Demand&Offer Matching Problem Statement section.
- Review the conclusions re. generic graph model (explain how classic "matching in graph" may not be directly applicable here)
- Discuss who can provide appropriate Golem Brass approach description for the doc?
- Address PJs request to reiterate the generic scenarios (without going into specific implementations)
Homework:
- MB: Write down description of the "Luxury Broker" scenario in the Use Cases section.
- PJ: Review the "Problem Statement" section, add comments.
- PR: Review the input on "Hybrid Implementation", add comments.
Participants: PJ, PR, PCh, MB
Summary: Discussed three main bidding models, "Tender model" suggested as most popular and practical. Homework assigned:
MB:
- Update "Demand&Offer Matching" with notes of discussed points.
- Update "Demand&Offer Specification Language" with a concept of Match Strength metric, which could be used in some implementations of matching algorithm for optimizations.
- Update "Market API" specification with Demand/Offer Remove & Unsubscribe operations.
- Reach out for R&D to ask for collaboration from Marcin Benke.
PR & PCh:
- Work on a concept of "hybrid" implementation of matching algorithm, place relevant notes in "Demand&Offer Matching" document.
Participants: PJ, PR, PCh, MB
Summary: Kick-off of Problem Statement discussion, results here: https://docs.google.com/document/d/1yTupuRsN9DKVrK1TPhM6dBxKCAPk0wCB8KxRf57ZkV4/edit?usp=sharing
Participants: MF, AW
Summary: TBC
Participants: JR, MF, AW
Summary: Review of pricing function ideas. Summary provided here: https://github.com/golemfactory/golem-architecture/wiki/Pricing-specifications
Participants: JR, MF, PR, MK (ITL)
Summary: Initial review of the definition of Execution Environments ("hosting" Plugins) for Golem. Summry of notes is here: https://github.com/golemfactory/golem-architecture/wiki/Execution-environments
Participants: JZ, JR, MF, AW, MB
Summary: Spontaneous catch-up/review chat with Julian. Recap of proposed pricing function model and price estimation request mechanism - idea of DSL dedicated to Golem Service Pricing considered.
Brainstorm of possible Demand&Offer matching implementations.
Participants: PJ (absent), JR, MB
Summary: Postponed
Participants: MF, AW, MB
Review session of the Provider metamodel.
Summary: Updates to https://docs.google.com/document/d/19MtGP6mQrS8BzOfE2jqVHSBPwYJtd96OvOw5PxuG61M/edit#heading=h.bdg90ydyrsrs Section 3.2.
Participants: PJ (absent), JR, MB
Summary: Postponed
Participants: JR, MB
Summary: Logical concepts of Golem Requestor Application architectures, persisted in https://docs.google.com/document/d/19MtGP6mQrS8BzOfE2jqVHSBPwYJtd96OvOw5PxuG61M/edit#heading=h.bdg90ydyrsrs Section 3.1.
Participants: MF, MB
Working session aimed at drafting component breakdown of Golem Requestor side implementation.
Summary: Notes persisted in https://docs.google.com/document/d/19MtGP6mQrS8BzOfE2jqVHSBPwYJtd96OvOw5PxuG61M/edit#heading=h.bdg90ydyrsrs Section 3.3.
Participants: PJ, JR, MB
Attempt to resolve the problem of comparing service prices.
Summary: Idea of including an estimation request in Demand. Updated https://docs.google.com/document/d/1tzMrhdBr9wiUXtSn1JO18MmIiP31dkMakdjStnF3eZY/edit# - "Service Price Estimating"
Participants: MF, MB
Working session aimed at drafting component breakdown of Golem Provider side implementation.
Summary: Notes persisted in https://docs.google.com/document/d/19MtGP6mQrS8BzOfE2jqVHSBPwYJtd96OvOw5PxuG61M/edit#heading=h.bdg90ydyrsrs Section 3.2.