A data sharing programme that has run unchanged for decades feels safe, but 30-year-old data systems quietly accumulate risk. Stale formats, forgotten rules, manual handling and security gaps build up for years before anyone notices. This article explains how legacy programmes decay across quality, security and governance, the warning signs an association should watch for, and what a modern, automated programme looks like instead.
When a data collection programme has run for 20, 25 or 30 years, almost nobody asks whether it should be modernised. The members are comfortable. The reports arrive on schedule. Internal systems have been wired to submit and receive data in exactly one format, so any change feels like a cascading rebuild rather than a simple upgrade. The instinct is understandable: if it isn't broken, don't fix it. But a legacy programme is rarely as healthy as it looks. The risks build slowly and invisibly, and they tend to surface only when something fails. In this post, we explain where the hidden risks of 30-year-old data systems sit and how to recognise them.
Why "if it isn't broken" takes hold
The word that describes it best is comfort, and the second is complacency. After years of running the same way, members and association executives like the format, like the data and accept the latency. Every internal pipeline has been configured around the existing programme, so change is never just a software swap. "Change" feels daunting: data flows have to be rebuilt, new fields have to be populated, and years of history may need backloading into a new format. On top of that, every change usually has to be agreed by all participants, which is slow and difficult. So the programme keeps running, not because it is the best option, but because changing it looks like work and risk.
The trouble is that a legacy programme can also repel the very participants it needs. Templates designed in the 1980s or 1990s to read the ERPs of the day now force a modern system to export data in an obsolete shape - an old spreadsheet, a flat table full of redundant columns, or short numeric codes standing in for plain words. Everyone who has been in the programme for 20 years knows that "200" means "local government". A newcomer has no idea. The programme feels antiquated, and significant new players hesitate to join. That is the opposite of what an association wants, because the value of the dataset depends on broad participation. This signalling problem is the same one we explored in interactive dashboards versus raw data: people judge a platform against the modern software they use every day.
How a legacy programme decays
The risks of a long-running programme split loosely across three fronts - quality, security and governance. They are described separately below, but in practice they decay together.
Governance drifts out of date
Two kinds of drift are common. The first is regulatory: laws on antitrust and the exchange of data between competitors do not change often, but they do change. A programme running in self-governing mode, where nothing is ever revised, can quietly fall out of step with current regulation. The second is operational. Programmes are usually governed by rules internal to the association, and those rules fade as people move on. When a new person inherits the reporting task at a participant, they may not know that only retail sales should be submitted. They start sending wholesale shipments to dealers, or distribution-centre movements, and record those as "sales". The dataset is no longer comparing like with like, and its value falls for everyone.
The rules fade until someone breaks them
Governance is not a one-time setup. Unless the rules are logged, adopted and communicated regularly, participants forget them. We regularly hear of breaches that begin innocently: a participant turns up at an industry expo advertising a number-one position and a market share percentage drawn from the programme. Using shared data for marketing is almost always explicitly prohibited, and it can take the association stepping in to have the material pulled down. The best-run programmes we have seen avoid this not by luck but by making their rules explicit and reinforcing them constantly.
Security gaps open up
Older programmes are run by people, because automation and cloud computing were not available when they began. Manual handling introduces human error: the wrong version of a file, a stale dataset, or an operator sending confidential data to the wrong recipient. We know of a case where a programme's administrator accidentally sent the entire database of every participant's sales (in a programme meant to be closed) to a single participant. The recipient happened to have a strong code of ethics and deleted the file, notifying only the administrator who was responsible for the mistake. But this mistake could have destroyed the programme in an instant, because the hard-earned trust of the participants would have been lost completely. Many of these gaps trace straight back to weak governance - unclear rules about who may share data with whom. The technical layer is what certification against a standard like ISO 27001 actually evidences, and it is the foundation of how we approach data and security.
The data itself goes stale
As technology advances, equipment gets smaller, lighter and more precise, and old segmentation stops making sense. Picture grading filters by hole diameter: "five millimetres and smaller" was once a sensible bucket, but as precision improves you need five to three, then three to one, then measurements in microns. It is like grading cars by top speed a century ago - "fifty and over" is meaningless when the fastest now exceed 400 km/h. A participant trying to read a 20-year trend cannot compare like with like, because the categories themselves have aged out of relevance. This is the same data-quality theme we set out in why industry market share data is never the full picture: the framework matters as much as the numbers.
Three warning signs an association should look for
When an association finally evaluates whether to modernise, three signs stand out:
- Human intervention. A legacy programme always has a human component, however small. As long as it exists, the programme is one wrong click away from a confidential file reaching the wrong participant. The ideal is no human in the day-to-day loop at all.
- Rigid, code-laden formats. If participants must submit data in one fixed shape full of redundant columns and opaque codes, that is a red flag. On the output side, a legacy system usually produces a single file with the latest month only - no trend lines, no bulk historical export, often no visual interface at all.
- A vendor who is not a data specialist. Many programmes were built in-house or handed to a trusted third party such as an accountant. Accountants are ethical and trusted, but they are not specialists in industry data collection or the antitrust nuances around it. We recently spoke with someone running a programme with a 1-person-firm operating it, whose partner had remarked, with audible frustration: "Oh, you're doing those monthly statistics again!" That one line revealed the bigger picture: an entire industry trusting its most prized asset - its sales data - to a process run on a private laptop in the evening, on a kitchen counter, with no security or governance controls behind it whatsoever.
The real barriers to change are human
When the warning signs are clear, the obstacles to acting on them are rarely technical. The first is people. When someone has used the same system for 20 years, their job is easy and predictable, and a new system feels like risk and effort, even when it would make their work simpler. Change management and the psychology of learning something new matter more here than any feature comparison.
The second barrier is harder to talk about. Some poorly run programmes produce numbers that are far from the truth, and a participant can have a vested interest in keeping it that way. We have sat across from a participant who told us plainly that they did not want more competitors joining the programme - because a fuller dataset would reveal a lower, truer market share than the inflated figure they had been reporting to their overseas parent for years. There is little anyone can do about that. It is worth naming, though, because it shows that a broken programme sometimes survives precisely because increased accuracy would be inconvenient.
Modernising without the disruption
The biggest fear about modernising is the work involved, and it is usually overstated. Most often, the transition is close to flicking a switch for the association, because a specialist provider can act as project manager. The association supplies a list of participating members and their contacts, and the provider spins the programme up, handles the communications and manages the migration. A simple safeguard keeps the association in control: every message is approved before it goes out, and a copy of every message is retained for the record.
The sequence itself is straightforward:
- It starts by recovering as much historical data as possible from the incumbent. Sometimes that is easy; sometimes the incumbent is unhelpful, or simply never archived the data, which is why an association should always hold its own backup.
- Recovered data is then checked for compatibility - legacy records are often lower resolution than a modern specification, but they remain usable.
- From there, the switch can be a timed hard cut-over, or the two systems can run in parallel so participants build confidence before the old contract lapses.
Crucially, the new system can be tailored to mirror the incumbent's input and output formats exactly, so participants upload the same file and receive the same report. The change becomes almost invisible to them, which removes one of the largest sources of resistance. Our view of how that transition should run is set out in our approach.
There is one more protection every association should have in place, regardless of vendor: a full-resolution master copy of its own data, held in trust down to model, brand and location detail. That custody must sit with an employed general manager or chief executive who is neither a board member nor a participant, because participants must never see the unredacted data. Without it, the retirement of a single key person can leave an entire programme stranded.
Certifications close the due diligence loop
A busy association does not have the time or expertise to audit a vendor's operation in depth. External certifications let it close that loop quickly:
- ISO 9001, the quality management standard, signals an obsession with accuracy - on a data collection programme the margin should be "+/-0", because the data is either perfect or it is rubbish.
- ISO 27001, the information security standard, with SOC 2 as a rough equivalent, signals that submitted data is protected in transit and at rest, against loss and against attack. PowerStats holds both.
- A third standard is emerging: ISO 42001, for AI management systems. As providers embed AI into their platforms, certification signals that the AI is governed through build, testing and deployment rather than bolted on as a gimmick - and that a human always verifies its output before anything is released.
The hidden cost of a cheap programme - and what good looks like
A legacy programme almost always looks cheaper on the invoice, and that comparison is misleading. The cheapness is exactly what erodes the value. A vendor locked into pricing set 20 years ago, on a programme that was never their core business, has little incentive to maintain quality or provide timely support. The bigger costs sit elsewhere: poor decisions made on untrustworthy data, the association's own staff time spent chasing submissions and fixing errors, and the analyst hours lost at every member organisation forced to wrangle data that arrives one clunky month at a time. That delay pushes insight to the board later and forces decisions on stale information - a cost a modern system can compress many times over.
A well-run programme is the opposite, and it is worth describing because it is what associations should aspire to. It feels light and almost invisible. Participants spend little or no time submitting, because the system can accept data automatically. Output is tailored to what each member wants, without giving anyone an advantage. The rules are clearly defined and communicated automatically. Contracts are modern, with real protections and current privacy rights built in (unlike legacy agreements, which often cannot even be located in signed form!). Above all, the day-to-day collection and reporting runs on deterministic code, on autopilot, with people reserved for the decisions that genuinely need them.
That is the real giveaway of a good programme: it runs without anyone having to touch it.
Key takeaways
- A legacy programme decays across quality, security and governance at once, and the decline is usually invisible until something fails.
- Forgotten rules are a real risk: when institutional knowledge fades, participants quietly start submitting the wrong class of data or misusing shared figures.
- Manual handling is the root of most security incidents, and a single misdirected file can destroy years of hard-won trust.
- Three warning signs matter most - human intervention in the day-to-day, rigid code-laden formats, and a vendor who is not a data specialist.
- Modernising is far less disruptive than it looks when a specialist runs the migration and mirrors the old formats, and ISO 9001, ISO 27001 and ISO 42001 let an association vet a vendor quickly.
- The "cheap" legacy option is the expensive one once poor decisions, wasted staff time and delay are counted.
Frequently asked questions
How do I know if our industry data programme has fallen behind?
Look for three signs. First, a person is involved in the day-to-day running rather than just governance decisions. Second, data goes in and comes out in a single rigid format, full of codes, with no trend views or bulk historical export. Third, the vendor is not a specialist in industry data collection. Any one of these is a prompt to review the programme.
Is moving to a new data platform as disruptive as it sounds?
Usually not. A specialist provider can act as project manager, so the association mainly supplies its member list and approves communications. Historical data is recovered and checked, the new system can be set up to mirror the old input and output formats, and the two systems can run in parallel until participants are confident. For most participants, the file they upload and the report they receive barely change.
What certifications should we ask a data provider to hold?
ISO 9001 evidences a quality management system and a commitment to accuracy. ISO 27001, or SOC 2 as a rough equivalent, evidences information security across the operation. ISO 42001 is the emerging standard for governing AI management systems, which matters as providers add AI features. Because these are externally audited, they let an association close due diligence quickly.
What happens to 20 years of history when we modernise?
It can be preserved. Historical data is recovered from the incumbent or the association's own backup and ingested into the new system, even where it is lower resolution. If segmentation needs updating, participants can either reload history under the new labels or fix the old categories at a cut-over date and apply the new structure from then on. The trend line does not have to break.
Talk to us about modernising your programme
If your data programme has run unchanged for a long time, the risks described here are worth a closer look - before a key person retires or an incident forces the question. PowerStats runs modern, automated, independently certified data programmes and can manage the migration on your behalf, mirroring what your participants already know. Contact us to talk through what modernising would involve for your industry.
