The project process

What actually should be done in order to add value?


The first step is scoping the project, it basically involves the questions

• What is the problem?
• What data is available?
• Input from key stakeholders
• Align input with data needs
• How much data is needed if data needs to be labeled etc?
• Assess the feasibility of data and stakeholder wishes
• Given the data what is the best way to solve the problem?
• Establish what the deliverables are.

When the scoping is done we aim to deliver a timeline and project planning.

The goal is to consolidate all available and required data and prepare it for modeling.

For some projects there will be a need to label data in order to train a model to do a specific task, in most cases, this is done by the client with support from us in terms of tooling, etc. In general, we do not have the required domain knowledge to do the labeling correctly.

Steps:

• Identify sources for required data
• Curate and validate data sources
• Perform syntactic quality check
• Data cleanliness assessment

Deliverables:

• Consolidated data specification
• ETL procedure for getting data into the model

The data collection, cleaning, and organizing step varies a lot from customer to customer and project to project. In the "simplest" case we basically just provide a data specification and agree on how the data should be transferred with us doing a simple data inspection to see that the data is ok and can go into the model before starting to train a model. In a very "complex" case where the customer may not have in-house staff to do it, we will do all of the work.

Create a Model based on requirements and data capable of delivering desired deliverables.


It involves:

• Import data into the modeling framework
• Explore possible models to find the best fit for the problem
• Select the best model and train it fully
• Evaluate the model performance

The deliverable is a fitness model and the other deliverables agreed upon during model scoping ex. recipe for refitting the model based on new data.

Create a Model based on requirements and data capable of delivering desired deliverables.


During the scoping, we determine what deliverables there should be together with the stakeholders. So during the delivery, we use the model to create those deliverables ex. insights in the form of a presentation, an interactive webpage, a rest API, or a docker container.


Then based on the sort of deliverables we provide it to the client together with the necessary support to utilize it fully.