Both the Prudential Regulation Authority (the PRA) and the Financial Conduct Authority (the FCA) rely on data for supervisory purposes and Nikhil Rathi, Chief Executive of the FCA, has said that “Data is the lifeblood of a modern regulator and in the next five years, we expect to become as much a regulator of data as a financial one” [1].
The FCA’s data strategy is part of its vision for the use of data and analytics and the strategy is centred on making better use of data to spot and stop harm faster. Data management and analytics are key parts of that. Data Science Units have been established to work with FCA sector experts to analyse risk, triage cases and automate processes. The intention is that harm will be detected more quickly in order to protect consumers.
Some data is from publicly-available sources but the FCA requires firms to submit data through its regulatory reporting platform, supplemented by information provided in response to supervisory requests. Regulatory reporting is currently focused on quantitative data, consistent with measuring and use of metrics, both of which the FCA has mentioned as part of firms’ monitoring work under the Consumer Duty. However, quantitative data makes up only part of the information firms use.
Qualitative information is needed too – for instance: formal and informal feedback from customers and others; the results of consumer research, including focus groups and testing; observations by Compliance or Internal Audit in the course of their review work; the results of root cause analysis work in relation to complaints. Firms produce qualitative information as part of their own MI and this is likely to increase as outcomes monitoring is reinforced under the Consumer Duty. The FCA has said that firms need to consider developing new data to evidence whether intended outcomes under the Duty are being met and that will almost certainly involve qualitative information as well as quantitative data.
It isn’t clear, at present, whether the FCA intends to use qualitative information as part of its data analytics work and, if so, how extensive that will be and how it will be done. Provided that it’s used appropriately, qualitative information would give the FCA a more complete picture of a firm: it can’t supervise appropriately if it hasn’t got the right data and the right information. My view is that, over time, the FCA will want to receive a wider range of information from firms – perhaps extending to board packs and board minutes from firms that don’t currently provide them, as well as qualitative information – and it will apply data analytics to that information. I expect that some qualitative information is already subject to data analysis but we do not have a clear line of sight into the data and information the FCA uses.
It’s easier to analyse quantitative data than qualitative information. Quantitative data provides a clear metric, whereas a metric has to be derived from qualitative data or a measure or value has to be determined. Natural language processing allows for interrogation and analysis but data sets need to be defined and parameters need to be set and I’m not aware of any recognised standardisation across both firms and regulators so far. The risk is that qualitative information received from different firms isn’t consistent and so can’t – properly – be compared or even evaluated in a meaningful way. For instance:
- Each firm will generate firm-specific qualitative information it needs by reference to its business, rather than a standardised set of information being produced by all firms.
- The context in which qualitative information has been prepared, and the intended use of it, will differ between firms.
- Qualitative information is likely to be specific to the products and services provided by a firm and its target market and customer and cohort profiles.
- What the FCA is looking for and using might be different from what firms in general or particular firms produce.
It follows that regulators could reach misleading conclusions about a firm if they rely on data analytics when reviewing qualitative information, exclude (or not take account of) it because it’s hard to fit into a data analytics model or operate a model that uses a different data set or reference points from the data set or reference points used by firms. Identifying concerns – and identifying outliers, which regulators are known to do and often act on – therefore runs a significant risk of being flawed.
I have little doubt that models are being generated and algorithms are being written to interrogate qualitative information in order to spot potential harm to consumers and markets. These do, though, need to take account of the limitations of that information. One approach might seem to be standardising the qualitative information firms must report to the FCA – for instance, by using a reporting template – but that’s unlikely to work for all firms, even those within the same sector. And using a standard template for reporting won’t standardise responses and still runs the risk that context critical to a firm’s business won’t be captured. Unless these points are addressed, incorrect conclusions are likely to be drawn about firms and the outcomes received by customers.
Until we have clarity on what information is being analysed and how the results are being used, it’s difficult to know what, if any, action firms should take. However, I suggest that firms do the following:
- Assume that all information they provide to regulators will potentially be regarded as ‘data’ that can be analysed – i.e. it isn’t just regulatory returns and numbers in spreadsheets that should be considered as ‘data’.
- Keep careful records of all qualitative information that’s provided to regulators, whether or not that’s in the form of reporting or responding to information requests.
- Ensure that all responses to information requests and other materials sent to regulators set out clearly background and context and any assumptions, qualifications, limitations on application or use and so forth. A covering note explaining any general points and context should also be provided.
- Look for any signs that information has been misinterpreted or taken out of context – or if a firm is being told, unexpectedly, that it’s an outlier.
This article is intended to provide general information about current and expected topics and perspectives that might be of interest. It does not provide or constitute, or purport to provide or constitute, advice relevant to any particular circumstances. Legal or other professional advice relevant to any particular circumstances should always be sought.
[1] Speech by Nikhil Rathi, Chief Executive of the FCA, on 16 June 2022, delivered at the Dutch Authority for the Financial Markets 20th anniversary seminar.