Iris: Display ingestion state for lecture slide upload

[yassinsws]

Motivation and Context

For Now we send a lecture for ingestion but there is no other UI that shows the state of the Ingestion. That's why this PR is created to add the Ingestion States for Lectures and Lecture Units.

Description

I added one column in the manage lectures view that shows whether a lecture is ingested and what is it's State.
I added a button to ingest one lecture Unit manually that also shows the ingestion State of the lecture unit.

Checklist

General

• I tested all changes and their related features with all corresponding user types on a test server.
• Language: I followed the guidelines for inclusive, diversity-sensitive, and appreciative language.
• I chose a title conforming to the naming conventions for pull requests.

Server

• Important: I implemented the changes with a very good performance and prevented too many (unnecessary) database calls.
• I strictly followed the server coding and design guidelines.
• I added multiple integration tests (Spring) related to the features (with a high test coverage).
• I added pre-authorization annotations according to the guidelines and checked the course groups for all new REST Calls (security).
• I documented the Java code using JavaDoc style.

Client

• Important: I implemented the changes with a very good performance, prevented too many (unnecessary) REST calls and made sure the UI is responsive, even with large data.
• I strictly followed the client coding and design guidelines.
• Following the theming guidelines, I specified colors only in the theming variable files and checked that the changes look consistent in both the light and the dark theme.
• I added multiple integration tests (Jest) related to the features (with a high test coverage), while following the test guidelines.
• I added authorities to all new routes and checked the course groups for displaying navigation elements (links, buttons).
• I documented the TypeScript code using JSDoc style.
• I added multiple screenshots/screencasts of my UI changes.
• I translated all newly inserted strings into English and German.

Screenshots :
Lecture Management view:

In case Pyris is Down the Ingestion State will give an error:

Lecture Unit View:

In case Pyris is Down all lecture units will display an error:

Steps for Testing

Prerequisites:

• 1 Instructor
• 1 Course with Iris Lecture ingestion enabled

1. Log in to Artemis
2. Navigate to Course Administration to enable Iris Lecture Ingestion
3. Upload an AttachmentUnit ( the pdf file attached here should do)
4. Send One Attachment Unit to Ingestion
5. The Icon Should change according to the phase it is in when you refresh the page.
6. Check if the Ingestion State is shown on the client side.
   blankpdf.pdf

Testserver States

Note

These badges show the state of the test servers.
Green = Currently available, Red = Currently locked
Click on the badges to get to the test servers.

Review Progress

Performance Review

• I (as a reviewer) confirm that the client changes (in particular related to REST calls and UI responsiveness) are implemented with a very good performance
• I (as a reviewer) confirm that the server changes (in particular related to database calls) are implemented with a very good performance

Code Review

• Code Review 1
• Code Review 2

Manual Tests

• Test 1
• Test 2

Test Coverage

Client

Class/File	Line Coverage	Confirmation (assert/expect)
attachmentUnit.model.ts	100%	✅
lecture.model.ts	100%	✅
lecture-unit-management.component.ts	84.61%	✅
lecture.component.ts	93.27%	✅
lecture.service.ts	97.08%	✅
course-overview.service.ts	92.64%	✅

Server

Could not generate code coverage here due to flaky tests failing constantly.

Client

Class/File	Line Coverage	Confirmation (assert/expect)
attachmentUnit.model.ts	100%	✅
lecture.model.ts	100%	✅
lecture-unit-management.component.ts	90.9%	✅
lectureUnit.service.ts	63.29%	✅
lecture.component.ts	93.27%	✅
lecture.service.ts	97.08%	✅
course-overview.service.ts	92.64%	✅

Server

|------------|--------------:|---------------------:|
| PyrisWebhookService.java | 82% | ✅ |
| LectureRepository.java | 86% | ✅ |
| LectureService.java | 100% | ✅ |
| LectureUnitService.java | 83% | ✅ |
| LectureResource.java | 90% | ✅ |
| LectureUnitResource.java | 80% | ✅ |

Summary by CodeRabbit

• New Features

  • Introduced ingestion state management for attachment units, enhancing tracking during the ingestion process.
  • Added functionality for directly ingesting lecture units and managing lecture state retrieval through new API endpoints.
  • Improved error handling in webhook services, allowing for better management of ingestion states and failures.

• Bug Fixes

  • Streamlined response handling in lecture ingestion methods to ensure consistent feedback.

• Refactor

  • Simplified method signatures and logic across various services for improved clarity and maintainability.

• Tests

  • Expanded test coverage for ingestion processes, ensuring robust error handling and state management.

[coderabbitai]

Walkthrough

The recent changes enhance the ingestion management system within the application by introducing structured state tracking for attachment units and related entities. Key improvements include the addition of a pyrisIngestionState field, modifications to service methods for improved error handling, and the introduction of new methods for managing lecture unit ingestion and status retrieval.

Changes

Files	Change Summary
src/main/java/.../AttachmentUnit.java	Introduced pyrisIngestionState field with getter/setter methods for tracking ingestion states.
src/main/java/.../LectureService.java	Modified ingestLecturesInPyris method to return void instead of boolean, focusing on execution without feedback.
src/main/java/.../LectureUnitService.java	Added ingestLectureUnitInPyris method for handling individual lecture unit ingestion.
src/main/java/.../PyrisConnectorService.java	Enhanced webhook handling with method renaming and improved error management. Added methods for retrieving ingestion states.
src/main/java/.../PyrisWebhookService.java	Updated to manage AttachmentUnit states and modified method signatures for improved job status updates.
src/main/java/.../PublicPyrisStatusUpdateResource.java	Introduced new methods for retrieving ingestion states of lectures and lecture units, with error handling.
src/test/javascript/spec/component/lecture-unit/lecture-unit-management.component.spec.ts	Updated tests to include new services and validate ingestion handling logic, ensuring robust test coverage for new functionalities.
src/test/javascript/spec/component/lecture/lecture.component.spec.ts	Removed references to IrisSettingsService and updated method names to reflect changes in the LectureService.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant LectureUnitResource
    participant LectureUnitService
    participant PyrisConnectorService

    User->>LectureUnitResource: Trigger ingestLectureUnit
    LectureUnitResource->>LectureUnitService: ingestLectureUnitInPyris(lectureUnitId)
    LectureUnitService->>PyrisConnectorService: addLectureUnitToPyrisDB(lectureUnit)
    PyrisConnectorService-->>LectureUnitService: Acknowledge ingestion
    LectureUnitService-->>LectureUnitResource: Return success response
    LectureUnitResource-->>User: Display success notification

Loading

sequenceDiagram
    participant User
    participant LectureUnitManagementComponent

    User->>LectureUnitManagementComponent: Click ingest button
    LectureUnitManagementComponent->>LectureUnitManagementComponent: updateIngestionStates()
    LectureUnitManagementComponent->>PyrisConnectorService: getLectureUnitIngestionState(lectureUnitId)
    PyrisConnectorService-->>LectureUnitManagementComponent: Return ingestion state
    LectureUnitManagementComponent-->>User: Update UI with new state

Loading

---

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

• Review comments: Directly reply to a review comment made by CodeRabbit. Example:
  • I pushed a fix in commit <commit_id>, please review it.
  • Generate unit testing code for this file.
  • Open a follow-up GitHub issue for this discussion.
• Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
  • @coderabbitai generate unit testing code for this file.
  • @coderabbitai modularize this function.
• PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
  • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
  • @coderabbitai read src/utils.ts and generate unit testing code.
  • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
  • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

• @coderabbitai pause to pause the reviews on a PR.
• @coderabbitai resume to resume the paused reviews.
• @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
• @coderabbitai full review to do a full review from scratch and review all the files again.
• @coderabbitai summary to regenerate the summary of the PR.
• @coderabbitai resolve resolve all the CodeRabbit review comments.
• @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
• @coderabbitai help to get help.

Other keywords and placeholders

• Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
• Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
• Add @coderabbitai or @coderabbitai title anywhere in the PR title to generate the title automatically.

Documentation and Community

• Visit our Documentation for detailed information on how to use CodeRabbit.
• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[yassinsws]

@HawKhiem I see you were using my course for the testing. The first file is an actual lecture slides that's why it takes too long to ingest that one. You can retry now I erased it and replaced it with the blank pdf that doesn't take time ( like the second file in your screenshot).
On the other hand the user is not supposed to be waiting for the ingestion to be done as it normally takes some time. That's why the done state is only shown when the user refreshes the page ( to send another request to iris and check if it's actually done, otherwise it wouldn't be efficient).

[HawKhiem]

I tested again on TS3. Everything now works smoothly, it shows when the ingestion is done and when I reload the page, the states of each file are displayed correctly!

[edkaya]

Retested on ts3, works as expected

[MichaelOwenDyer]

After the latest changes it looks good to me now