sensenet/sn-taskmanagement

A standalone .Net component for managing long-running background tasks in any application.

Welcome to sensenet

sensenet is a content repository with API first approach packed with a full featured permission system, preview and collaboration tools.

A basic setup of sensenet has three top-level parts:

• A content repository that is the storage and service layer
• An application that uses the content of the connected content repository through API calls
• A sensenet admin surface (only in SNaaS) that helps you carry out common content management tasks

Everything is content

Content is the basic block for storing information in sensenet. A content can be any kind of data, user, document, workspace, memo, task, and more. Using content items everywhere unlocks a great deal of exceptional features, making your experience as a user more seamless, and your job as a developer a lot easier.

Licensing

SNaaS (sensenet as a service)

In this model, sensenet content repositories live in our cloud infrastructure.

Advantages

• no installation required (easy onboarding)
• patches and upgrades are taken care of by us
• no hosting related tasks
• central admin surface
• flexible pricing plans (based on # of contents, requests, and users)

on-prem

For the terms of on-prem licensing please contact our sales team.

Resources

Check out the links below to get further information:

• Website: https://www.sensenet.com/
• Documentation: https://docs.sensenet.com/
• Main chat channel: https://gitter.im/SenseNet/sensenet
• Community support: http://stackoverflow.com/questions/tagged/sensenet

Contributing

All kinds of contributions are welcome! We are happy if you have an idea, bugfix or feature request to share with others. Please check out our Contribution guide for details.

sn-taskmanagement's Issues

Upgrade task manager projects to .Net Core

• upgrade agent and web server to netcore (partially: agent-web authentication is a different issue)
• set new taskmanager urls during builds

Rethink the task management architecture

Our task management architecture is built on console app agents and executors which is a bit too complicated to maintain and test. Rethink the architecture with the following points in mind:

• deployment: operators should be able to deploy the whole system or parts of it anywhere (on-prem, cloud)
• scalability: executing tasks should be scalable (being able to execute more tasks by adding more resources)
• progress indication: the current system is capable of providing real-time progress info during task execution. Keep that feature.
• logging: accessible executor and agent logs are essential.
• authentication: every element of the system should be able to work separately and connect to each other in a secure way.

Tech to check

• Azure functions in a hybrid environment
• Azure functions and SignalR?

Publish task management images

How to get document

Hi team, how can I get document or example for taskmanagement, it looks links are broken

TLS issue - Taskmanager does not work

Deploy task management in a Docker image

After upgrading to .Net Core, we can deploy the task management web application and agent+executors in Docker images so that we can scale agents easily.

• deploy task management web in a Docker image
• deploy task management agent and preview generator in a Docker image

[BUG]: taskmanagement table columns are too short

Exception:
Extended Properties: Messages - SqlException: String or binary data would be truncated in table 'databasename.dbo.TaskEvents', column 'Machine'.

Database created with this script:
https://github.com/SenseNet/sn-taskmanagement/blob/master/src/TaskManagementWeb/Data/Scripts/Install_TaskDatabase_Schema.sql

Modernize standalone and snaas client authentication

Currently the two versions of the task management code (standalone and snaas) have their code differences inline, per feature. We should create a provider for implementing those features so that merging becomes easier.

Also: modernize client authentication because currently the task management web cannot authenticate correctly when sending finalize requests back to the repository (ApplicationHandler class, GetHttpClient method).

Solution

• a new interface in the SenseNet.TaskManagement.Core project for the possible customization points (e.g. SetAuthenticationAsync).
• implement it locally for standalone mode and in a new SNaaS.TaskManagement project for snaas.

Store and use API keys for applications using task manager

Currently we have to configure authentication values (clients/secrets) for client applications. It would be better if we set an API key when registering an application. This would let us connect to the task manager from repositories dynamically.

• add a new nvarchar column to the db (scripts, patch, model) for holding a JSON object (multiple api keys) There is already an AppData column that holds a json object, we can extend that.
• pass the API key on to the agent and executor to allow them to connect to the repository
• use it on the web, during the finalize request
• refresh the tokens periodically: repositories can do this using maintenance tasks.
• handle SNaaS: do NOT register api keys in that case? Do we need a different mechanism?

TODO in a different issue: restrict client apps (white list of allowed repositories?).

Unnecessary ping request may block the task deletion from the DB if there was no finalization URL given

File: AgentHub.cs
Method: TaskFinished

The ApplicationHandler.SendPingRequest tries to ping the webserver regardless I provided a finalization URL or not.
If I start a new task without a finalization URL and valid application URL (because I don't have/need one), the ping will block the removement of the task.

Error during preview generation! No preview available.

Preview generation throws error on some files when call "Generate" through admin ui.,
For example "DinnersCookBook" in demo repository.

The two related logs, a verbose and a warning contains the same info: WARNING #61cba2b0-cf9e-45f7-b571-9aa641a9468e: Task#4852 execution TERMINATED with error. Result:134, task type: AsposePreviewGenerator, agent: taskagent-taskmanager-int-sensenet-cloud-6ddb7fdcf7-gfk9g-Agent#1

Task management web scalability for SignalR

Previously the task management web app was scalable using a SQL backplane for SignalR. The new .Net Core SignalR best practice is to use Redis as the message broker.

• add Redis backplane feature to taskman web app
• deployment and maintenance

Convert TaskManagement projects to .Net Standard

Convert projects to .Net Standard, or at least .Net Framework 4.6.1.

Consider modernizing the agent tools and the web app too to minimize the dlls to deploy. (postponed: revisit when modernizing the whole module for the cloud)

Small readme typo + sentence discussion

The readme contains a small typo which I would like to fix with a PR:

perform more tasks at the samwe time

Also a question about the following line:

It is a robust and scalable solution that is extendable with 3rd party task executors designed for solving atomic background tasks [...]

The way I read it the component has support for 3rd party task executors that are designed for solving atomic background tasks. Is this true, or is the component itself designed for solving atomic background tasks? If the latter I would like to re-write that sentence to make it more clear.

Upgrade task management projects to .net 6

The goal is to make task management projects (web, agent) work in the new build and deployment environment.

Task management in the cloud

This is a discussion story that will initiate smaller stories in the future.

Task management is a standalone product that can be installed on one or more virtual machines, so it can work with a sensenet-powered application in the cloud. However the architecture can be improved to be a more modern, cost-effective member of the sensenet app family in the cloud.

Task management web app

Currently the task management app (which behaves like a service) is an Asp.Net web application. We should consider converting it to a more scalable, cloud-friendly app service.

Scaling tasks

The task management web app is responsible for orchestrating tasks, notifying agents about a new job, channeling errors back to the portal. In a cloud environment scaling can be outsourced to the infrastructure: we do not need agents, we should be able to start any number of tasks and the infrastructure will provide the resources (based on the subscription plan).

Proposal: the agent and the service can be eliminated from the infrastructure.
ToRead: Azure Functions

Exclusivity

We have to keep the feature that makes sure that tasks are started by a single executor. The locking mechanism (through the db) is already in place, we do not want to change that.

Deployment

• Currently we deploy the task management app as a WebPI package, this should be improved to a more cloud-friendly solution.
• Deploying executors to the sensenet web app through NuGet is also something we need to change.

Monitoring

In sensenet there is already a monitoring page based on SignalR that is able to process and display status messages from executors, we do not want to change that.

The central task management application may be capable of displaying a central monitor page in the future, or should we direct these status messages to a 3rd party monitoring system in the cloud?

Task management authentication

Currently the task management component works fine if it operates behind a firewall, in a local network. If we want to make it accessible from outside as a service, we have to improve the authentication options.

Mandatory

• task web ---> sensenet authentication (finalizing tasks): configure secrets for every sn application (load auth info from snover)
• agent/executor --> sensenet authentication: configure secrets for every sn application (load auth info from snover)
• test the new auth flow with real secrets

Later, out of scope for now

• sensenet ---> task web authentication (register tasks, monitor)
• agent <---> task web authentication (SignalR)