9 Signs Your Skills Development ProgrammeIs Measuring Reach Instead of Impact

A programme can train 2,000 people, issue 2,000 certificates and deliver 50 workshops.

Those numbers may all be accurate.

They still do not tell us what changed.

Did participants use what they learned? Did their confidence improve? Did their behaviour change? Did they progress into work, further learning or enterprise? Did the programme remove a barrier that existed before?

This distinction matters because reach is not the same as impact.

My concern is not that organisations count participants. They should. Attendance, completion and delivery data are useful.

The problem begins when those numbers are treated as the final proof that a skills development programme worked.

What is skills development impact?

Skills development impact is the meaningful change that can reasonably be linked to a programme, beyond the fact that training was delivered. Depending on the programme, that change may involve knowledge, behaviour, confidence, work readiness, employment, enterprise, financial capability, further learning or improved organisational performance.

South African evidence suggests that training evaluation is still difficult in practice. A University of Cape Town study published in the South African Journal of Human Resource Management found a substantial gap between training-evaluation theory and what sampled corporates actually did. Reaction-level data were collected more often than learning, behaviour, results or return-on-investment measures. Source: SciELO

That is important because the easiest thing to count is not always the most important thing to know.

1. You report attendance as the headline outcome

“People reached” is a useful output.

It tells us scale.

It does not tell us change.

If your annual report says:

• 5,000 people trained
• 80 workshops delivered
• 4,200 certificates issued

I still want to ask:

What happened because of the programme?

Trialogue’s 2025 South African monitoring and evaluation research found that outputs remain widely measured. Ninety-one percent of surveyed companies measured CSI project outputs across all programmes. Many also reported measuring outcomes and impact, but Trialogue noted how difficult genuine impact measurement is in practice. Source: Trialogue

Reach should be part of the story.

It should not become the whole story.

2. Certificates are treated as proof of success

A certificate can be valuable.

But it proves a specific thing: a participant completed, attended or demonstrated something according to the certificate’s rules.

It does not automatically prove that the participant:

• found work
• started a business
• improved job performance
• changed behaviour
• became more financially capable
• progressed into further learning
• sustained the learning after the programme

The distinction is especially important in skills development impact measurement.

If the certificate becomes the final outcome, the programme may stop asking harder questions.

Those harder questions are where learning begins.

3. You mainly measure participant satisfaction

“Did you enjoy the workshop?”

“Was the trainer engaging?”

“Would you recommend this programme?”

These questions can be useful.

They measure experience.

They do not necessarily measure learning or change.

The South African training-evaluation study found that reaction-level data were collected more frequently than deeper measures of learning, behaviour or results in the sampled organisations. The researchers argued that training evaluation often remains too shallow or inconsistent. Source: SciELO

A participant can love a workshop and learn very little.

A participant can find a workshop challenging and still use it months later.

Satisfaction belongs in the picture, but it should not be mistaken for impact.

4. You never established a meaningful baseline

If you do not know where participants started, it becomes difficult to understand what changed.

A baseline does not need to become a 40-page research exercise.

It might include:

• current knowledge
• current confidence
• employment status
• business activity
• digital capability
• financial behaviour
• access to opportunity
• relevant habits
• current support networks

The baseline should match the purpose of the programme.

For a work-readiness programme, attendance at a workshop is not the same as movement towards employment.

For an entrepreneurship programme, completing a Business Model Canvas is not the same as testing a customer proposition or earning revenue.

Measure what the programme is actually trying to move.

5. You stop measuring when the training stops

This is one of the most common design problems.

The final session happens.

The facilitator submits attendance.

Certificates are issued.

The report is written.

The programme is declared complete.

But some of the most important changes happen later.

Did participants apply the learning after one month? Three months? Six months?

Did they retain the skill?

Did they progress?

Did they encounter a barrier the programme could have anticipated?

Outcome-focused funding models in South Africa are increasingly putting more weight on what happens after enrolment. The Jobs Boost Outcomes Fund, for example, ties funding milestones to job placement and retention, not only programme entry. Source: Trialogue

Not every programme should copy that model.

But every programme should understand the principle:

the end of training is not necessarily the end of the outcome journey.

6. Participant voice is weak in programme design

This is the sign I care about most.

A programme can have excellent dashboards and still misunderstand the people it serves.

Trialogue’s 2025 research found that surveyed companies collaborated more with implementing partners than directly with beneficiaries in monitoring and evaluation. Beneficiary involvement was stronger in interviews and feedback than in programme design or M&E methodology. Source: Trialogue

That should make us ask:

Are participants only sources of data?

Or are they sources of knowledge?

Coming from a previously disadvantaged community myself and having experienced programmes first as a learner and later as a practitioner and programme leader, I see this difference very clearly.

People often know exactly where the friction is.

They know when transport is a barrier.

They know when the examples do not fit their lives.

They know when the facilitator is moving too fast.

They know when the “opportunity pathway” on a slide does not exist in practice.

Listen before you measure.

7. The reporting template decides what matters

Reporting templates can be helpful.

They create consistency.

They can also quietly shape the programme around what the donor finds easy to report.

If the template asks only:

• attendance
• demographics
• completion
• workshop count

then implementing teams can become very good at generating those numbers.

The danger is that the programme begins serving the report instead of the participant.

A strong reporting framework should contain enough structure for accountability and enough flexibility to capture what the programme is learning.

Sometimes the most useful insight was not predicted at proposal stage.

That is not a failure of planning.

It may be evidence that the programme is actually paying attention.

8. Findings do not change the programme

Measurement has little value if nobody uses it.

Trialogue found that many surveyed companies and nonprofits reported using evaluation findings for planning, strategy and programme revision. That is encouraging because measurement should create a feedback loop, not just a report. Source: Trialogue

Ask a simple question:

What did we change because of what we learned?

Maybe onboarding was too complicated.

Maybe facilitators needed more support.

Maybe participants needed longer practice time.

Maybe transport affected attendance.

Maybe the course content was useful but the opportunity pathway was weak.

If the same problems appear in every cohort and the programme never changes, the measurement system is collecting information without creating learning.

9. There is no pathway beyond the intervention

A programme can teach useful skills and still leave participants stranded.

The missing piece is often connection.

What does the skill connect to?

Employment?

A market?

Further learning?

An internship?

A business opportunity?

A professional network?

A more advanced programme?

If the organisation cannot explain what the programme is helping participants move towards, then impact measurement will also remain vague.

That is why programme design and measurement should never be separated completely.

You cannot measure meaningful movement if the programme has never defined what movement should look like.

A practical measurement framework

I prefer a simple sequence:

1. Inputs

What resources went into the programme?

2. Activities

What did the programme actually do?

3. Outputs

What was delivered and who participated?

4. Outcomes

What changed in the short or medium term?

5. Impact

What broader, longer-term change can reasonably be linked to the intervention?

Then add one more question:

6. Learning

What should the programme do differently next time?

This last step matters.

A measurement system should not only prove value to funders.

It should improve value for participants.

Related support

For programme design and measurement support, explore Skills Development Consulting, Social Impact Programme Consulting and Programme Implementation Consulting.

For funder and CSI work, visit Funders & CSI. For programme evidence, see the ReCha case study.

Related reading: the seven questions every CSI funder should ask before funding another training programme and seven reasons a certificate of completion does not prove training impact.