Product and project management for terraphim
License: GNU Affero General Public License v3.0
Product and project management for terraphim
Driven by Earthly CI: Create bucket:
earthly --secret-file-path /Users/alexandermikhalev/rust_code/terraphim/op_args -i +publish --region="eu-west-2" --args="s3 mb s3://project-manager-terraphim-kg-ci"
earthly +run will copy relevant data into bucket. It's no longer possible to create a bucket with public acl via CLI.
Wireframe for UI, but we should clean it up, now it has a lot of excessive elements.
Knowledge graph [[INCOSE]], role [[system operator]], the test article is this. By the way, the HTML code does not contain the article, it won't work. What do we do in such cases?
Problem. For people, it is challenging to create structured notes, especially using templates like inputs-activities-outputs. Also, to make a proper knowledge graph, we need frontmatter and other enriched metadata, and it's even more difficult. LLMs and other AI solutions are unreliable and have little value without a knowledge graph. As stated in the blog post, we should use explicit user feedback to improve search results, and we cannot do that without the knowledge graph.
Context. When we tag content, we should use wider terms from the standard (controlled dictionary), not terms from the content itself (it's good when they coincide, but it may be only in the title and headers). Otherwise, it will be unmanageable. We will use ChatGPT-generated summaries to contrast the ideas from the text. Data sources on the user side are siloed. We unify them in the process model by tagging them and making them discoverable with the meta.
Solution. We map narrower terms to wider terms and lead users through the checklist to control the quality of notes. Each wider term has an out-of-the-box context to help enrich the note:
The user renames generic relations (specific relations rename automatically), answers checklist questions, and sends these notes to the Logseq journal (Terraphim also copies it to the clipboard). Still, the open question is how we use those notes as user feedback to improve the search.
Features.
Benefits.
Deep reading. Together with ChatGPT summaries, it provides insight generation and critical thinking tools and helps develop original ideas about the texts. Practically the user can:
These things require further feature development, but this is a good start.
Develop the methodology (and realize it in Atomic Server) of Terraphim Domains and explain how they relate to Terraphim Roles. Terraphim Roles and Domains should affect domain topic suggestion, autocompletion, and selection. The initial idea is that the Terraphim Role covers several Domains, which are process models (causal models in terms of >>>Pearl, J. (2019) ‘The seven tools of causal inference, with reflections on machine learning’, Communications of the ACM, 62(3), pp. 54–60. Available at: https://doi.org/10.1145/3241036.) >>>)
from the relevant standard. So, we'll need both domain models because they structure the knowledge graph and the Terraphim Roles that structure the query. The collection of Domains is a haystack of relevant concepts and relationships.
The feature is a part of the Topical search and article ingestion user flow (use case).
It guides the user's search prompt, offering domain topic keywords from the SE digital process model terminology. The user can, for example, type in a synonym, and Terraphim will substitute it for the proper broader term. I imagine it now as a two-step prompting feature:
UX and UI are from the Bing browser. I like it very much.
1. Study Terraphim Desktop setup on systems.tf and terraphim.ai webpages and understand how it works.
(a) Systems.tf should contain the demo video (problem, context, solution, benefits, trial instructions) and all necessary information to perform this end-to-end user flow.
(b) Discourse Terraphim should contain the topic that addresses Q&A for this user flow.
2. Download installation packages for Terraphim desktop and browser extension, Atomic Server, GitHubDesktop, and Logseq from the websites. Check the control sums (optional) and install all of them.
(a) Links to installation packages should be available on the downloads page.
(b) Readme.txt with installation and setup instructions should be available at terraphim.ai and systems.tf websites.
3. Add a demo graph (INCOSE-Systems-Engineering) or actual domain graph in GitHub Desktop.
(a) a demo video should explain why and how we use domain graphs, including the technical side.
(b) Should be user instructions on how to add a demo graph in GitHub Desktop.
4. Open Logseq and add the domain graph from the local folder of the GitHub repository.
(a) Should be user instructions on how to add the graph and why the user needs it (short explanation on semantic interoperability).
5. Perform {{query "type:: [[TerraphimRole]]"}} in Logseq and get the list of Terraphim roles available in the graph (domain graph pages). Open the Terraphim role page and export it as JSON to the local folder (MacBook “Downloads”).
(a) A short tutorial and an article on role mechanisms should be available.
6. Open the Terraphim desktop app and import (drag and drop) the JSON file with the Terraphim role. Select the local domain graph folder for Terraphim Desktop, import the graph, and add it as a local haystack. Export domain graph to a browser extension.
(a) a short video tutorial on performing Terraphim role export and import should be available, and an explanation of the haystack mechanism.
7. Select a local folder with files (projects, library) as a haystack.
8. Open systems.tf website or working resource and test that the web extension tags the Terraphim role concepts with the command “Ground concepts”. The browser extension shows the counter for concepts on the page, the breakdown, and orphaned/omitted concepts. Possible action: add the webpage to the Terraphim Cortex haystack.
9. Select the type of highlight for the Terraphim role. Ground concepts feature - links to graph pages or backlinks.
10. Select whether inputs, activities, and outputs will be shown according to the selected Terraphim role for the Ground concepts feature. Format [[Concept: input/output/activity]], for example, [[System concepts: input]].
11. Search: selecting haystacks - local folder with sources and the Logseq graph.
12. Search: prompt autocompletion from graph concepts.
13. Search: results showing the source, metadata (obligatory), and summarization of the paragraph where the search term is found.
14. Search: open the webpage from the link to the source (only for webpages).
15. Search: add the webpage to the Terrahim Cortex haystack.
16. Command “Annotate” from the context menu creates the node in the domain graph. It adds Zotero-like metadata from the webpage and additional property terraphimrole:: [[System operator (or whatever that is selected)]]. The page's name appears in the clipboard so you can find and open it in the Logseq app and the Terraphim role in the Logseq side pane.
17. On the side pane, three sections with a checklist appear: Do general and specific relationship names make sense? Do narrower terms contradict or match broader terms or put them into question? Is there a cause-effect chain between inputs, activities, and outputs? After each question, there is a blank field for user entry. Notes are connected to a paragraph, and once the annotated section is sent to Logseq (copied to clipboard), you can paste it with all your notes under the title and the front matter.
18. After the whole article is annotated, a user writes his overall notes and tags synonyms.
Probably one of the broadest audience in Microsoft ecosystem
UI, UX, and overall implementation of the search prompt, including the syntax and rules of selection of composed domain topics and terms.
Deletes everything
Implement multiple terms and synonyms matching that support the search itself and domain topic suggestion, autocompletion, and selection.
In the beginning, the user has a templated Business function node(s), for example:
type:: [[Business function]]
source:: [[@Digital Systems Engineering Process Model Version 1]]
documentation:: As stated in ISO/IEC/IEEE 15288, [6.2.5.1] The purpose of the Quality Management process is to assure that products, services and implementations of the quality management process meet organizational and project quality objectives and achieve customer satisfaction. See detailed description in the INCOSE Handbook v.4, page 156.
inputs:: [[Organization strategic plan]], [[Quality assurance plan]], [[Quality assurance report]], [[Quality assurance evaluation report]], [[Customer satisfaction inputs]]
outputs:: [[Quality management plan]], [[Quality management guidelines]], [[QM corrective actions]], [[Quality management report]], [[Quality management evaluation report]], [[Quality management record]]
activities:: [[Plan quality management]], [[Assess quality management]], [[Perform quality management corrective action and preventive action]]
At the end of the flow, the user has the enriched Business function node and the Terraphim role. The Terraphim role has key Terraphim terms and synonyms to them:
https://system-operator.s3.eu-west-2.amazonaws.com/term_to_id.json. The user needs terms and synonyms to perform role-based search further.
Overall, 30 business function nodes exist in the graph. This use case can cover any of them or any composition of individual business function nodes but no more than five nodes. The graph also contains hundreds of nodes with bibliographical sources, like this:
tags:: [[Chief Executive Officer]], [[Implementation Strategy]], [[Product Data]], [[Product Development]], [[Total Quality Management]]
date:: 2016
publisher:: Springer
title:: @Product lifecycle management
book-title:: Product Lifecycle Management (Volume 2)
pages:: 1–35
item-type:: [[bookSection]]
original-title:: Product lifecycle management
authors:: [[John Stark]]
library-catalog:: Google Scholar
links:: [Local library](zotero://select/library/items/XL8M2VLA), [Web library](https://www.zotero.org/users/6520516/items/XL8M2VLA)
All bibliographical nodes start with @. It also can contain the field abstract like this:
- [[Abstract]]
- Causal decision theory adopts principles of rational choice that attend to an act’s consequences. It maintains that an account of rational choice must use causality to identify the considerations that make a choice rational., Given a set of options constituting a decision problem, decision theory recommends an option that maximizes utility, that is, an option whose utility equals or exceeds the utility of every other option. It evaluates an option’s utility by calculating the option's expected utility. It uses probabilities and utilities of an option’s possible outcomes to define its expected utility. The probabilities depend on the option. Causal decision theory takes the dependence to be causal rather than merely evidential., This essay explains causal decision theory, reviews its history, describes current research in causal decision theory, and surveys its philosophical foundations. The literature on causal decision theory is vast, and this essay covers only a portion.
The graph also has 99 nodes with type:: [[Business process]]. Node names of these pages are also used in the search.
The user performs this task in several steps:
1. The user selects the business function (search topic) and creates a prompt to find the relevant bibliographical sources.
Terraphim autocompletes the prompt with activities and outputs and proposes to add the inputs from the business function. Each activity, output, or input is a block in a search prompt and can be deleted with one click. The user can also add any term to the prompt. Search results should include the nodes with any of the terms, but the term here can consist of several words, like [[Plan quality management]]; Terraphim should propose terms autocompletion from inputs, outputs or activities, or business process names. Visual guidance is Bing search:
Suggestions go in the drop-down. The search prompt should start with "Topic:"; it should not be a blank search prompt field. It also should indicate that the user now creates a Terraphim role (the search lens). For Terraphim role modification, it will be another use case.
2. The user views the search results, selects relevant sources for further topic search terms extension with synonyms from bibliographical sources, and creates a blank Terraphim role page.
Terraphim shows a complete search query above and produces the list of bibliography nodes that match this query (the user performs this as no role search). Search is performed over all the content of the bibliography node. It must be an exact match with one or more search terms. The more search terms coincide, the higher the relevance. There should be a button to export search results into CSV or JSON format. Search results should be presented in the table - title, year (date), authors, and highlighted keywords that were found.
For example, for the example above, Terraphim should produce this list:
The user can "Terraphim it" each source selection. Terraphim proposes to create a new Terraphim role or modify the existing. It proposes to name it (by default, the name is safe as a business function), checks for duplicates and creates a relevant Logseq page. The user adds the Terraphim role template on this page and opens it in the sidebar.
When the user opens the bibliographical source, the proper Logseq page (see 1)-4) above) should open together with the business function page (topical page). In the Logseq PDF annotation plugin, the user can attach and annotate the PDF with the Logseq highlights plugin. As the user annotates the PDF, he adds synonyms and related concepts to the page in the Logseq:
synonyms:: operate the system, operation of system, system maintains key functions, system operationally effective, maintenance actions, breakdown in services, monitor the services, monitor system performance, sustain service, identify analyze operational problems, analyze operational problems, service life extension program, SLEP, manage the migration between systems
relatedconcepts:: [[Disposal]], [[Transition]], [[Maintenance]], [[Validation]], [[Knowledge Management]], [[Change management]], [[Concept of operations (ConOps)]], [[Measurement]]
Once the user opens ht bibliography node, Terraphim adds the line with the topic name in the frontmatter:
relatedconcepts:: [[Operation]]
terraphimrole:: [[System operator]]
Also, the user adds synonyms for activities, input, or outputs:
Prepare for operation
type:: [[Business process]]
terraphimrole:: [[System operator]]
source:: [[@Digital Systems Engineering Process Model Version 1]]
documentation:: See detailed description in the INCOSE Handbook v.4, page 96.
synonyms:: sustain a pool of operators,
Also, the user adds an attribute for terraphimrole:: [[System operator]].
Currently, this feature is for the bibliography, which is only in the graph. Later, we will add external sources (articles from web pages) processing.
All node names and synonyms fields with attribute terraphimrole:: should be processed in the search for this role as described in the 1.
3. Add the relevant SFIA skills and synonyms.
Goes the same way as with bibliography sources, but for the SFIA skills catalog.
sfiaskills:: [[IT infrastructure]], [[Database administration]], [[Application support]], [[Security operations]], [[Certification scheme operation]], [[Storage management]], [[Network support]], [[Service catalogue management]], [[Technology service management]], [[Asset management]], [[Demand management]], [[Measurement SFIA]], [[Sustainability]], [[Continuity management]], [[Information security]], [[Acceptance testing]], [[Organisational capability development]], [[Systems installation and removal]], [[Facilities management]], [[Service level management]], [[Availability management]], [[Capacity management]], [[Incident management]], [[Problem management]], [[Change control]], [[Service acceptance]]
Terraphim looks for the topic name ("quality management") in the skills description, provides the list, opens the Logseq page, adds synonyms, but also generates the Logseq pages for the selected skills:
IT infrastructure
type:: [[skill]]
terraphimrole:: [[System operator]]
source:: [[@The global skills and competency framework for a digital world]]
documentation:: Deploying, configuring and operating IT Infrastructure.
sfiacode:: [[ITOP]]
sfiaskill:: [[IT infrastructure]]
sfiacategory:: [[Delivery and operation]]
sfiasubcategory:: [[Technology management]]
sfiaguidancenotes:: IT infrastructure components include, but are not limited to, physical devices, virtual resources, infrastructure-related software, middleware, network services and data storage. Infrastructure components may be on-premises, outsourced, or provisioned as cloud services. Activities may include — but are not limited to: preparing for new or changed services to meet defined needs of organisational users or providers; maintaining and enhancing the IT infrastructure and infrastructure components, including task automation via tools and coding; managing and applying software updates; building and managing systems and components in virtualised and cloud computing environments; monitoring the performance of systems and services related to their contribution to organisation performance, security and sustainability.
sfialevel1description:: Contributes, under supervision, to routine infrastructure operation. Gains understanding of infrastructure components and services by following the activities of experienced colleagues.
sfialevel2description:: Carries out routine operational procedures, including the execution of specified automation tools/scripts. Amends existing automation tasks under supervision to gain a basic understanding of the scripting language/automation tools. Contributes to maintenance and installation. Monitors and reports on infrastructure performance to enable service delivery. Resolves issues or refers to others for assistance.
sfialevel3description:: Provisions/installs, configures and maintains infrastructure services and components. Monitors, measures and reports on infrastructure load, performance and security events. Identifies operational issues and contributes to their resolution. Carries out agreed operational procedures, including backup/restore, using supplied infrastructure tools and scripts. Carries out agreed system software maintenance tasks. Automates routine system administration tasks to specifications using standard tools and basic scripting.
sfialevel4description:: Provides technical expertise to enable the correct application of operational procedures. Contributes to the planning and implementation of infrastructure maintenance and updates. Implements agreed infrastructure changes and maintenance routines. Uses infrastructure management tools to determine load and performance statistics. Configures tools and/or creates scripts to automate the provisioning, testing and deployment of new and changed infrastructure. Maintains operational procedures and checks that they are executed following agreed standards. Investigates and enables the resolution of operational issues. Provides reports and proposals for improvement, to specialists, users and managers.
sfialevel5description:: Provides technical leadership to optimise the performance of IT infrastructure. Investigates and manages the adoption of tools, techniques and processes (including automation) for the management of systems and services. Oversees the planning, installation, maintenance and acceptance of new and updated infrastructure components and infrastructure-based services. Aligns to service expectations, security requirements and other quality standards. Ensures that operational procedures and documentation are fit for purpose and kept up to date. Ensures that operational issues are identified, recorded, monitored and resolved. Provides appropriate status and other reports to specialists, users and managers.
When user types :config me new window shall pop up with config
Second largest audience
Follow CI integration for:
[ ] Terraphim Private cloud
[ ] terraphim firecracker-vm
[ ] terraphim project
[ ] terraphim system operator KG repository
The largest audience
Create a Logseq template and Terraphim workflow to work with a domain term. Especially how we create synonyms and how they should affect the ranking and autocompletion.
My crucial assumption, for now, is that people have three opportunities to improve their knowledge base. Student projects, downtime in the task at work, and job switching. Other times, they are stuck and lack a chance to change something.
Osterwalder business canvas model in Logseq for Terraphim. Put a link here.
Go-to-market plan in Word (Innovate UK).
Develop a publications plan.
Demo1: Job interview (customer meeting) prep (both CV and STAR-answers techniques prep)
Demo2: Certification exam prep (INCOSE CSEP)
Demo3: Performance review prep (SFIA demo)
Demo4: Knowledge base refactoring and migration
Demo5: Knowledge base quality metrics and performance
Demo6: Time capsule for future me
Demo7: Meaningful comments and question cues on someone's agenda, workshop program, website, speech, presentation, article, etc. (any piece of information we have on essential people)
Demo7: Logseq page refactoring using BFO
Demo8: RASIC matrix generation
Demo9: Skills-project matching - low priority
Demo10: Reminding what to do for the tasks that people perform rarely
Maria Keet makes a good point that can be our argument why we better than Dropbox Dash:
1.3.2 The lack of trust (or the angst) on what data went and go into the LLMs (the ‘could be your emails, googledocs, sharepoint files etc.’), that no-one was asked whether they consented to their content being grabbed for that purpose, and when some would have disapproved of inclusion if they could, there’s the powerlessness in that it seems one neither can opt out nor verify if one’s text was excluded if opt-out were to be possible.
So, when you search over Outlook or Google docs, you violate the consent of your collaborators. That is a good point for privacy-first search.
Terraphim Domain for INCOSE use case is a set of pages with a tag [[Business function]], a set of сфгыфддн related processes. One Terraphim role, for example, [[System operator]], can cover several such Domains, which I call Domains Collection. Domains Collection can be a haystack, and we'll need to figure out how to model in Atomic Server and how to display in Terraphim Search.
Collection of Terraphim Domains should affect the ranking relevance for domain topic suggestion, autocompletion, and selection.
Develop reading and taking into account Terraphim roles during the search. Terraphim role should be created and updated in Terraphim Cloud and Desktop once it is changed in the Logseq repository. It should affect domain topic suggestion, autocompletion, and selection.
Develop and test the Logseq template to describe Terraphim roles. Prepare training and explanatory videos for Terraphim roles.
User segment:
Development teams working in meritocratic or corporate environments where regular performance assessment exists or is planned to be introduced. People are ready to spend time and effort to provide verifiable track records because it will affect their careers. They are prepared to use structured documents to perform assessments, not oral ad hoc discussions over the performance standard, where the boss decides everything.
User story:
As a project or product manager of the development team, I want to perform a fair, transparent (explainable), and evidence-based assessment process with the PDCA improvement cycle.
As an engineer and team member of the development team, I want to keep a proper track record of my achievements that do not need redoing for each project or job, that can be easily supported with evidence from different data sources, and publish this track record for my manager, customer, or peers for review so they corroborate it like on LinkedIn, but more privately and profoundly.
Problem: Asymmetry of the information about individual performance (manager cannot understand whether the engineer's job done is significant) and team performance (engineer cannot know what other people on the team did and their level).
For project and product managers, getting concise but precise descriptions of achievements from qualified engineers and supporting them with verifiable evidence can be challenging, especially when engineers switch jobs. And it is tiring to make engineers provide such depictions, especially in verbal reports. So, managers only give their credentials (which they formulate themselves) to those they have worked with for a long time and refrain from hiring or referring prominent candidates based on the story these candidates provide, as they cannot verify the validity of the job experience. Also, people change over time for the worse, so even when the manager knows someone from years ago, he still can be reluctant to give his recommendations because this is risky.
For engineers, it is difficult to understand what managers want when asking broad questions like "Tell me about yourself." Many qualifications of such engineers stay hidden because nobody asked specific questions during the interview or review; engineers rarely possess oratory skills and don't want to spend time acquiring them. But they want a fair and transparent assessment process.
And even more specifically - how do engineers interpret the feedback, and how do managers ensure engineers receive and process the feedback? Because the concept of behavior feedback is vague, see my post "Feedback went wrong".
Even worse - if we present at the CEO speech, how do we consider what her feedback is we should be considering? If we listen to the podcast where we hear the advice to change something, why is it not feedback for us? It should be.
In this use case, we use the talent digital shadow to hashtag the received feedback to help the user interpret it correctly and the author of the feedback to ensure the feedback is valuable.
Situation (context):
The team does not use commercial or corporate HRM solutions to manage the assessment process. They may want to buy such a solution later but want to test if the process works first. So it is small and medium-sized companies.
The team now does not value the existing assessment process and executes it only to meet corporate requirements and considers time spent on it wasted.
People cannot use the results of the assessment after they switch the job. And with an average position switching every 2-3 years, that is a lot of wasted effort over a 40-year career.
As the process is worthless, it is performed the day before the deadline with minimal effort, so performance records produce no factual information to the managers above.
As records contain no actual information about the team member's skills and experience, the decisions about salary raises or promotions or otherwise keeping people are heavily biased toward people with whom the decision-makers have personal knowledge. Project and product managers cannot argue to save people with evidence in their hands; they need to have track records.
When you switch jobs, you don't transfer your reputation because your track record is scattered across LinkedIn, verbal evidence from your colleagues and bosses, your contracts, and, most importantly, not searchable for others even if you manage to collect it.
Solution:
Benefit:
End-to-end user flow:
It's a part of the due diligence demo flow:
Demo1: Job interview (customer meeting) prep (both CV and STAR-answers techniques prep)
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.