Cloud Hosting

Introduction to Internet of Things (IOT):

Internet of Things (IoT) represents the fourth-generation technology that facilitates the connection and transformation of products into smart, intelligent and communicative entities. IoT has already established its footprint in various business verticals such as medical, heath care, automobile, and industrial applications. IoT empowers the collection, analysis, and transmission of information across various networks, encompassing both server and edge devices. This information can then undergo further processing and distribution to multiple inter-connected devices through cloud connectivity.

IoT is used in the Oil and Gas Industry for two basic reasons: First low power design, a fundamental requirement for intrinsically safe products, Second two-way wireless communication. These two advantages are a boon for the products used in Oil and Gas industries. The only challenge is for the product design to meet the hazardous location certification.

An intrinsic safe certification is mandatory for any device placed in hazardous locations. The certification code depends on the type of protection, zone, and the region where the product shall be installed.

In the North American and Canadian markets, the area classification is done in three classes:

Class I: Location where flammable gases and vapors are present.

Class II: Location where combustible dust is present.

Class III: Location where flying is present.

The hazardous area is further divided into two divisions, based upon the probability that a dangerous fuel to air mixture will occur or not.

Dvision-1: Location is where there is a high probability (by underwriting standards) that an explosive concentration of gas or vapor is present during normal operation of the plant.

Division-2: Location is where there is a very low probability that the flammable material is present in the explosive concentration during normal operation of the plant; so, an explosive concentration is expected only in case of a failure of the plant containment system.

The GROUP is also one of the meaningful nomenclatures of the hazardous area terms.

The four gas groups were created so that electrical equipment intended to be used in hazardous (classified) locations could be rated for families of gases and vapors and tested with a designated worst-case gas/air mixture to cover the entire group.

The temperature class definitions are used to designate the maximum operating temperatures on the surface of the equipment, which should not exceed the ignition temperature of the surrounding atmosphere.

Areas classified per NEC Article 505 are divided into three zones based on the probability of an ignitable concentration being present, rather than into two divisions as per NEC article 501. Areas that would be classified division 1 are further divided into zone 0 and zone 1. A zone 0 area is more likely to contain an ignitable atmosphere than zone 1 area. Division 2 and zone 2 areas are essentially equivalent.

Zone-0: Presence of ignitable concentration of combustible gases and vapors continuously, or for long periods of time.

Zone-1: Intermittent hazard may be present.

Zone-2: Hazard will be present under abnormal conditions.

IoT-based products can be designed for various applications, a few of them are listed below:

A typical block diagram of the IoT application is shown below:

Figure 1: IOT Block Diagram

An IoT product might consist of a battery as a power source or can be powered externally from either 9V ~ 36V DC supply available in the process control applications or 110/230Vac input.

The microcontroller can be selected based on the applications, power consumption, and the peripheral requirements. The microcontroller converts the analog signal to digital and based on the configuration can send the data on wired/wireless to the remote station. Analog signal conditioning stands as a pivotal component of the product, bridging the connection between the sensor and facilitating the conversion of analog signals for compatibility with the microcontroller. The Bluetooth interface suggested in the example is due to its wide acceptance and low power consumption. The wireless interface depends on the end-application of the product.

The electronics design of an IoT product for a hazardous location is very complex and needs a careful selection of the architecture and base components as compared to the IoT developed for commercial applications. In case the IoT is for a hazardous location, the product must be intrinsically safe and should not cause an explosion under fault conditions. The product architecture should be designed considering various mechanical, and electronics requirements as defined in the IEC 60079 standards, certification requirements and the functional specifications.

Power Source: This is one of the main elements in an IoT-based product. Battery selection should meet the overall power budget of the product, followed by the battery lifetime. In case of intrinsic safety, special consideration is required for where the battery in charged. IEC 60079-11 clause 7.4 provide details for the type of battery and its construction details. Separation distance from the battery and electrical interface should be done as per Table-5 of IEC 60079-11. If the battery is used in the compartment, sufficient ventilation must be provided to ensure that no dangerous gas accumulation occurs during discharge or inactivity periods. In scenarios where IoT operates on DC power sources such as 9~36Vdc (nominal 24Vdc), the selection of power supply barrier protection becomes a critical consideration, particularly when catering to intrinsic safety norms. This necessitates a thorough analysis of the products prerequisites and the mandatory certifications. Adding to the complexity is the existence of IoT devices functioning on 230Vac, demands intrinsic safe calculations and certifications aligned with Um = 250V.

Microcontroller: Its central processing unit of the IoT product. The architecture of the microcontroller, power, and clock frequency processing must be carefully selected for a particular application. The Analog to Digital Conversion (ADC) part of the microcontroller should be selected based on the required accuracy, update rate, and resolution. Microcontroller should have enough sleep modes so that the power is optimally utilized for IoT applications and should have sufficient memory/peripheral interface to meet the product specifications.

Analog Signal Conditioning: The front-end block should meet the intrinsic safe requirements as per the IEC 60079 standards and should also protect the product from EMI-EMC testing. Barrier circuit should provide enough isolation for meeting the spark-gap ignition requirements and impedance requirement of the transducer. Also, along with the safety requirements, the designer should ensure that extracted sensor signal is not degraded from the excessive noise present in outside environment. All the sensors used for collecting data from the process parameters to the signal conditioning block must be certified for the particular zone.

Wireless Communications: There are various wireless options available for sending data from the IoT product to the sensor such as (6LOWPAN, ZigBEE, ZWave, Bluetooth, Wi-Fi, Wireless HART). Selection of a particular wireless interface requires knowledge of end application, RF-power, antenna, and protocol. Selection of the interface for a particular IoT application should be done keeping these basic things in mind:

In case of intrinsic safe applications, its important to note that the use of certified modules does not directly confer suitability for deployment in hazardous locations. The product must undergo fresh testing within an intrinsic safe lab to assess both quantifiable and non-quantifiable ffaults, along with spark testing. or the countable and non-countable faults and spark testing. The RF power transmitted from the devices should be limited as per Table-1x of IEC 60079-0.

When building IoT solutions for hazardous locations, special conditions relating to creepage and clearance, encapsulation, and separation distance must be carefully considered. Also, when battery and RF signals are used, its expected the designer should be aware of the applicable standards and limitation of these standards for such products.

With more than 25 years of experience in designing mission-critical and consumer-grade embedded hardware designs, eInfochips is well poised to make products which are smaller, faster, reliable, efficient, intelligent and economical. We have worked on developing complex embedded control systems for avionics and industrial solutions. At the same time, we have also developed portable and power efficient systems for wearables, medical devices, home automation and surveillance solutions.

eInfochips, as an Arrow company, has a strong ecosystem of manufacturing partners who can help right from electronic prototype design, manufacturing, production, and certification. eInfochips works closely with the contract manufacturers to make sure that the designs are optimized for testing (DFT) and manufacturing (DFM) to reduce design alterations on production transfer. To know more about this contact us.

einfochips can help product-based companies to develop intrinsically safe products and get the product certified by lab for various certifications like ATEX/IECEx/CSA.

Kartik Gandhi, currently serving in the capacity of Senior Director of Engineering, possesses a distinguished career spanning over two decades, marked by a profound expertise in fields including Business Analysis, Presales, and Embedded Systems. Throughout his professional journey, Mr. kartik has demonstrated his proficiency across diverse platforms, notably Qualcomm and NXP, and has contributed his talents to several esteemed product-based organizations.

Dr. Suraj Pardeshi has more than 20 years of experience in Research & Development, Product Design & Development, and testing. He has worked on various IoT-enabled platforms for Industrial applications. He has more than 15 publications in various National and International journals. He holds two Indian patents, Gold Medalist and Ph.D (Electrical) from M.S University, Vadodara.

See the article here:

Internet Of Things (IOT): Application In Hazardous Locations ... - Data Science Central

Read More..

Elke Rundensteiner, the William Smith Deans Professor in Computer Science and Founding Head of WPIs Data Science Program, recently received the InfoVis 20-Year Test-of-Time Award from the Institute of Electrical and Electronics Engineers (IEEE) for her pioneering work on data visualization and visual analytics in 2003.

This award honors articles published at previous IEEE VIS (Visual Identification System) conferences, in this case in 2003, that have withstood the test of time by remaining useful 20 years later and that have had significant impact and influence on future advances within and beyond the visualization community, according to the awards organizers. Award selection is based on measures such as the numbers of citations, the quality and influence of its ideas, and other criteria.

Rundensteiner and her team, which included the late computer science professor Matthew Ward and former PhD students Jing Yang and Wei Peng, are being honored for their work on interactive hierarchical dimension ordering, spacing, and filtering for the exploration of high-dimensional datasets.

I fondly remember my close research collaboration with my colleague Matt Ward over a 17-year time span from 1998 to 2014 that resulted in a series of 7 National Science Foundation (NSF) research grants and one National Security Agency (NSA) grant for our work at the intersection of visualization and data analytics, Rundensteiner said. This allowed us to collaborate with countless joint PhD students, contributing cutting-edge advances to the then-newly emerging area of visual analytics, which led to this inspiring award. Matt was not only a creative thinker at the forefront of his time, he was a supportive colleague and generous friend and remains a true inspiration for me.

According to the award selection committee, the work that Rundensteiner and her team undertook presents a thoughtful, elegant, and powerful approach to managing the complexities of high-dimensional data and reducing clutter in visualizations such as parallel coordinates. The teams research provided insight by clustering the dimensions of high-dimensional data sets into a hierarchical structure (instead of just clustering the data itself), which then can be exploited to make sense of this complex data more efficiently. The paper, Interactive heirarchical dimension ordering, spacing and filtering for exploration of high dimensional datasets, laid the groundwork for subsequent research and influenced the design of other tools and techniques, the award committee noted.

Citations to the original paper have increased over time, showing evidence of lasting value, and the ideas introduced in the work are still relevant today, the award committee wrote. The paper shows us how we can solve a problem through interactive visualization design and presents convincing options for future analysts and designers. These ideas underpin subsequent research on synthesizing new summary dimensions, contribute to contemporary thinking on explainability, and have influenced the design of many other high dimensional visualization tools and techniques.

Read this article:

Elke Rundensteiner Receives the Prestigious IEEE Test-of-Time ... - WPI News

Read More..

In the search for new physics, a new kind of scientist is bridging the gap between theory and experiment.

By Benjamin Nachman | October 13, 2023

Traditionally, many physicists have divided themselves into two tussling camps: the theorists and the experimentalists. Albert Einstein theorized general relativity, and Arthur Eddington observed it in action as bending starlight; Murray Gell-Mann and George Zweig thought up the idea of quarks, and Henry Kendall, Richard Taylor, Jerome Freidman, and their teams detected them.

In particle physics especially, the divide is stark. Consider the Higgs boson, proposed in 1964 and discovered in 2012. Since then, physicists have sought to scrutinize its properties, but theorists and experimentalists dont share Higgs data directly, and theyve spent years arguing over what to share and how to format it. (Theres now some consensus, although the going was rough.)

But theres a missing player in this dichotomy. Who, exactly, is facilitating the flow of data between theory and experiment?

Traditionally, the experimentalists filled this role, running the machines and looking at the data but in high-energy physics and many other subfields, theres too much data for this to be feasible. Researchers cant just eyeball a few events in the accelerator and come to conclusions; at the Large Hadron Collider, for instance, about a billion particle collisions happen per second, which sensors detect, process, and store in vast computing systems. And its not just quantity. All this data is outrageously complex, made more so by simulation.

In other words, these experiments produce more data than anyone could possibly analyze with traditional tools. And those tools are imperfect anyway, requiring researchers to boil down many complex events into just a handful of attributes say, the number of photons at a given energy. A lot of science gets left out.

In response to this conundrum, a growing movement in high-energy physics and other subfields, like nuclear physics and astrophysics, seeks to analyze data in its full complexity to let the data speak for itself. Experts in this area are using cutting-edge data science tools to decide which data to keep and which to discard, and to sniff out subtle patterns.

Machine learning, in particular, has allowed scientists to do what they couldnt before. For example, in the hunt for new particles, like those that might comprise dark matter, physicists dont look for single, impossible events. Instead, they look for events that happen more often than they should. This is a much harder task, requiring data-parsing at herculean scales, and machine learning has given physicists an edge.

Nowadays, the experimentalists who manage the control rooms of particle accelerators are seldom the ones developing the tools of machine learning. The former are certainly experts; they run colliders, after all. But in projects of such monumental scale, nobody can do it all, and specialization reigns. After the machines run, the data people step in.

The data people arent traditional theorists, and theyre not traditional experimentalists (though many identify as one or the other). But theyre here already, straddling different camps and fields, proving themselves invaluable to physics.

For now, this scrappy group has no clear name. They are data scientists or specialized physicists or statisticians, and they are chronically interdisciplinary. Its high time we recognize this group as distinct, with its own approaches, training regimens, and skills. (Its worth noting, too, data physics discreteness from computational physics. In computational physics, scientists use computing to cope with resource limitations; in data physics, scientists deal with data randomness, making statistics what you might call phystatistics a more vital piece of the equation.)

Naming delivers clout and legitimacy, and it shapes how future physicists are educated and funded. Many fields have fought to earn this recognition, like biological physics, sidelined for decades as an awkward meeting of two unlike sciences and now a full-fledged and vibrant subfield.

Its the data wranglers turn. I propose that we give these specialists a clear identity the data physicists. Unlike a traditional experimentalist, a data physicist probably wont have much hands-on experience with instrumentation. They probably won't spend time soldering together detector parts, a typical experience for experimentalists-in-training. And unlike a theorist, they may not have much experience with first-principles physics calculations, outside of coursework.

But the data physicist does have the core skills to understand and interrogate data complete with a strong foundation in data science, statistics, and machine learning as well as the computational and theoretical background to relate this data to underlying physical properties.

The data physicists have their work cut out for them, given the enormous amount of data being churned out by experiments in and beyond high-energy physics. Their efforts will, in turn, improve the development of new experimentation methods, which are today often developed from simpler, synthetic datasets that dont map perfectly to the real world.

But this data will go underutilized without a skilled cohort of scientists who can deftly handle it with new tools, like machine learning. In this sense, Im not merely arguing for name recognition. We need to identify and then train the next generation, to tackle the data we have right now.

How? First, we need the right degrees: Universities should develop programs explicitly for data physicists in graduate school. I expect the data physicist to have a strong physics background and extensive training in statistics,data science, and machine learning. Take my own path as a starting point: I studied computational aspects of particle theory as a masters student and took many courses in statistics as a PhD student, which led to naturally interdisciplinary research between physics and statistics/machine learning and between theorists and experimentalists.

The right education is a start, but the field also needs tenure-track positions and funding. There are promising signs, including new federal funding to help institutions launch Artificial Intelligence Institutes dedicated to advancing this research. But while investments like this fuel interdisciplinary research, they dont support new faculty not directly, at least. And if youre not at one of the big institutions that receive these funds, youre out of luck.

This is where small-scale funding must step in, including money for individual research groups, rather than for particular experiments. This is easier said than done, because a typical group grant, which a PI uses to fund themselves and a student or postdoc, forces applicants to adhere to the traditional divide: theory or experiment, or hogwash. The same goes for the Department of Energys prestigious Early Career Award there is no box to check for interdisciplinary data physics.

As tall an order as this funding is, it could be easier to achieve than a change in attitude. Physicists might well be famous for many of humanitys greatest discoveries, but theyre also notorious for their exclusionary, if not outright purist, suspicion of interdisciplinary science. Physics that borrows tools and draws inspiration from other fields from cells in biological physics, say, or from machine learning in data science is often dressed down as not real physics. This is wrong, of course, but its also a bad strategy: A great way to lose brilliant physicists is to scoff at them.

Not all are skeptical; far more, in fact, are excited. Within APS, the Topical Group on Data Science (GDS) is growing rapidly and might soon become a Division on Data Science, a reflection of the fields growing role in physics. My own excitement about working directly with data inspired me to become an experimentalist myself, although I realize now how restrictive that label was.

As available data grows, so does our need for data physicists. Lets start by calling them what they are. But then lets do the hard work: educating, training, and funding this brilliant new generation.

Benjamin Nachman is a Staff Scientist at Berkeley Lab, where he leads the Machine Learning for Fundamental Physics Group, and a Research Affiliate at the UC Berkeley Institute for Data Science. He is also a Secretary of the APS Topical Group on Data Science.

The author wishes to thank the Editor, Taryn MacKinney, for her work on this article, and David Shih for coining the term 'data physicist' at a recent Particle Physics Community Planning Exercise.

Read more from the original source:

Opinion: The Rise of the Data Physicist - American Physical Society

Read More..

NEW YORK, October 16, 2023--(BUSINESS WIRE)--Ducera Partners LLC ("Ducera"), a leading investment bank, and Growth Science Ventures, a data science focused venture capital firm, today announced the launch of Ducera Growth Ventures ("Ducera Growth").

Ducera Growth Ventures will focus on identifying, analyzing, and managing innovation-based venture capital investments in funds that include strategic corporate clients. The platform will be led by Michael Kramer, Founding Partner and Chief Executive Officer of Ducera, and Thomas Thurston, one of the worlds leading data scientists, Founder of Growth Science Ventures, and a Senior Advisor to Ducera.

Unlike traditional venture capital investing, Ducera Growth will combine Duceras investment banking expertise with Growth Sciences proprietary analytics and big data systems to identify unique early-stage growth companies. With an adherence to classic disruption theory and competitive threat regression analyses, the platform will deploy growth capital on behalf of its strategic corporate clients in future market leaders and next-generation companies that demonstrate the potential to produce new customers, reduce costs, and/or create new markets that are consistent with a clients long-term vision.

Michael Kramer, Chief Executive Officer of Ducera Partners, said, "Ducera continues to evolve as a full-service investment bank and strategic advisory firm, and we are focused on providing our clients with access to innovative solutions that we believe have the ability to add significant value to their businesses. Ducera Growth Ventures is an exciting new initiative that I believe will disrupt traditional venture capital investing and redefine how companies access and/or acquire innovative external technologies."

Thomas Thurston, Founder of Growth Science Ventures and Senior Advisor to Ducera, added, "Innovation is now happening at speeds, scales, and levels of complexity that only advanced computing can adequately analyze and make sense of. Yet with the right technological tools and a mastery of how to use them, companies can identify and capture growth from disruptive opportunities more rapidly and consistently than ever before. Utilizing data science and artificial intelligence presents a significant opportunity for companies to enhance their productivity and decision making in support of their organic and inorganic growth strategies. I am thrilled to partner with Michael to form Ducera Growth Ventures and look forward to working with a broad array of Duceras corporate clients in support of their venture investing interests."

Story continues

Mr. Thurston and Ducera previously launched the first of a six-part mini-series focused on how Ducera is using data science and artificial intelligence to advise clients in their development of corporate innovation and growth. Learn more by visiting: https://ducerapartners.com/news/thomas-thurston-partner-and-founder-of-growth-science-ventures-has-joined-the-firm-as-a-senior-advisor/

About Ducera Partners

Ducera Partners is a leading investment banking advisory practice with expertise in restructuring, strategic advisory, liability management, capital markets, wealth management, and growth capital. Since its founding in June 2015, Ducera Partners has advised on over $750 billion in transactions across various industries. Ducera Partners has offices in New York, Los Angeles, and Stamford. For more information about Ducera Partners, please visit http://www.ducerapartners.com.

About Ducera Growth Ventures

Ducera Growth Ventures focuses on identifying, analyzing, and managing innovation-based investments across a broad array of market segments and industries. The platform seeks to combine Duceras investment banking expertise with Growth Science Ventures proprietary analytics and big data to identify early-stage growth companies that have the potential to be successful over the long term. With an adherence to classic disruption theory and competitive threat regression analyses, Ducera Growth Ventures invests growth capital on behalf of its strategic corporate clients in future market leaders and next-generation companies that have the potential to produce new customers for Duceras clients, reduce costs, and/or create new markets that are consistent with a clients long-term vision. For more information about Ducera Growth Ventures, please visit http://www.ducerapartners.com

About Growth Science Ventures

Growth Science Ventures was founded by Thomas Thurston, one of the worlds leading data scientists. The firm utilizes data science to identify disruptive startups and counsels clients in connection with the development and launch of new products and services. For nearly 20 years Growth Science has continued to evolve its proprietary analytics, capabilities, and AI infrastructure through research collaborations with more than 60 of the world's largest, market-leading multinational companies spanning more than 1,000 market segments. For more information about Growth Science Ventures, please visit http://www.gsventures.com.

View source version on businesswire.com: https://www.businesswire.com/news/home/20231016545173/en/

Contacts

Mike GellerProsek Partnersmgeller@prosek.com

View post:

Ducera Partners and Growth Science Ventures Announce the Formation of Ducera Growth Ventures - Yahoo Finance

Read More..

Foreword from the Chair of Court

Data are critical to the work of a central bank. The Bank of England has long recognised this. Most recently, we defined decision-making informed by the best available data, analysis and intelligence as a timeless enabler of our mission. And, to deliver on that, in 2021 we made modernise the Banks ways of working a strategic priority for the years 2021 to 2024.

At the same time, the pace of innovation in data and analytics continues to increase, as the recent advances in the capabilities of large language models make clear. Every day, the Bank makes decisions that affect millions of the UKs people and businesses the Banks data and analytics capabilities support and power that decision-making process. It is therefore vital that we stand back and consider whether our data capabilities will remain fit for purpose in a rapidly changing world such that we deliver our timeless enabler and ultimately our mission.

To that end, in October 2022 the Banks Court of Directors commissioned its Independent Evaluation Office (IEO) to conduct an evaluation of the Banks use of data to support its policy objectives.

The IEOs report is clear. Overall, and despite many positive steps, looking forward the Bank must ensure that its data capabilities advance to match its ambition, especially as data and analytics best practice advances rapidly. While the Bank is not alone in facing this challenge, addressing it is strategically critical. The Bank will therefore need to set itself up for success by stepping up the pace of change, investing in its technology and people, and overcoming the barriers that will impede progress in a rapidly changing data and analytics landscape.

The IEOs recommendations provide a foundation for doing so. They make 10 detailed recommendations, grouped into three broad themes: committing to a clear vision for data and analytics, supported by a comprehensive strategy and effective governance; overcoming the institutional, cultural and technological barriers faced by organisations as they move to new and emerging data-centric ways of working to keep in step with a changing world; and ensuring the Banks staff have the support and skills they need.

At our 22 September meeting, Court welcomed the Banks commitment in taking forward these recommendations. We will monitor their implementation as part of the IEOs follow-up framework.

David Roberts, Chair of CourtOctober 2023

Data have long been at the heart of central banking. But the availability of data and the capabilities to draw insights from them have developed rapidly over the past decade or so. These changes, when coupled with expanding remits and global shocks, have created both opportunities and challenges for central banks. In that context, in October 2022 the Court of Directors (the Banks board) commissioned its IEO to conduct an evaluation of the Banks use of data to support its policy objectives.

In response to rapid change, central banks have innovated in multiple dimensions, from institutional structures to technological infrastructure, to new analytical methods and data sources. But, like many organisations, they have faced a range of challenges along the way, whether from legacy systems, established working practices or the practicalities of cloud migration.

The Bank of England has been on a similar journey to peer central banks. It made data a prominent feature of the 2014 One Bank strategy and in the strategic priorities for the next three years that it set out in 2021. It created the role of Chief Data Officer supported by an expanding team. It has developed a sequence of data strategies (Box A), founded on credible problem statements. It has rolled out new analytical and storage capabilities with associated training for staff. And, supported by its emerging centres of excellence, it has done pioneering analysis with new techniques and data sources with examples ranging from the use of machine learning to predict individual bank distress from regulatory returns and plausibility checking returns from regulated firms, through to tracking the macroeconomy at high frequency during the pandemic with unconventional measures of activity. footnote [1] footnote [2]

The Banks current data and analytics operating model devolves a large amount of responsibility for data and analytics to its business functions. The central Data and Analytics Transformation Directorate, led by the Chief Data Officer, is responsible for enabling those areas in their delivery of the central data strategy, which is half of one of the Banks seven strategic priorities for 202124. This model is currently in transition, partly in response to our evaluation and partly as a result of leadership change, with a new Chief Data Officer having started in role in April 2023.

Our evaluation took the overarching research question Is decision-making to support the Banks policy objectives informed by the best available data, analysis and intelligence, and can it expect to be so in the future?. We adapted this from the Banks timeless enabler on data, which was set out alongside the 202124 strategic priorities. We broke that down into four detailed areas of investigation, covering broad questions of strategy and governance and three detailed areas of data management: acquisition and preparation; storage and access; and analysis and dissemination. Our evidence gathering involved: conducting around 175 interviews, across the Bank and a range of other organisations, including peer central banks and regulators; a staff survey, complemented by targeted focus groups; and consulting an advisory group of senior Bank staff and, separately, two external expert advisors.

With best practice in data and analytics advancing rapidly, the Bank will need to step up the pace of change and associated investment if it is to take advantage of new opportunities. While progress has been made using a devolved operating model, data capabilities are inconsistent across the organisation and in some cases the current approach is sub-optimal. To progress further, management will need to systematically address a range of foundational technology and process issues and build the capabilities necessary to enable the Bank to take advantage of new data tools so it can be in the best position to deliver on the Bank's mission. We make 10 detailed recommendations, which we grouped into three broad themes: committing to a clear vision, supported by a comprehensive strategy and effective governance; breaking down institutional, cultural and technological barriers to keep in step with a changing world; and ensuring staff have the support and skills they need.

Theme 1: Agree a clear vision for data and analytics, supported by a comprehensive strategy and effective governance.

1. Agree and champion a vision for data use, matching funding to ambition.

2. Collaboratively design deliverable Bank-wide and local business area data strategies to meet measurable business outcomes.

3. Ensure governance structures can support the agreement, co-ordination and monitoring of data transformation, with clear accountability for delivery.

Theme 2: Break down institutional, cultural and technological barriers to keep the Bank's data and analytical practices in step with a changing world.

4. Improve day-to-day collaboration across the business on data and analytics.

5. Agree the approach to sharing data and analytics inside and outside the Bank.

6. Narrow the gap with modern data and analytics practices, with the most impactful initial step being a phased migration to cloud.

7. Systematically monitor and experiment with new approaches and technology for data and analytics.

Theme 3: Ensure staff have the support and skills they need to work effectively with data.

8. Embed common standards to make data and analysis easily discoverable and repeatable.

9. Provide staff with the easily accessible support and guidance they need across the data lifecycle.

10. Develop a comprehensive data skills strategy encompassing hiring, training, retention and role mix.

In addition, the Bank is currently taking a range of actions to strengthen key foundational enablers. Successful execution of these wider initiatives will be crucial to fully delivering the Banks data ambitions: i) improvements to the approach to setting organisational strategy, prioritisation and budgeting; ii) tackling technology obsolescence; and iii) strengthening of the Banks central services and change management capabilities. The appointment of an Executive Director to lead a new Change and Planning function, the delivery of the Central Services 2025 programme, and future iterations of the Banks wider talent strategy will contribute across these areas of focus.footnote [3]

The evaluation was conducted by a dedicated project team reporting directly to the Chair of Court.footnote [4] The IEO team benefited from feedback and challenge from a Bank-wide senior-level advisory group (including Bank Governors). David Craig (founder and former CEO, Refinitiv, former Head of Data and Analytics, LSEG, and Executive Fellow, London Business School) and Kanishka Bhattacharya (Expert Partner, Bain & Company, and Adjunct Associate Professor, London Business School) provided support and independent challenge to the team and reviewed and endorsed the findings in this report.

This report was approved for publication by the Chair of Court at the September 2023 Court meeting.

Data have long sat at the heart of central banking, including at the Bank of England. At least since the heyday of the gold standard, monetary policy makers have drawn on data to determine the stance of monetary policy. The Banks Quarterly Bulletin, the Banks flagship publication from its introduction in 1960 through to the 1993 launch of the Inflation Report, offered a detailed commentary on economic and financial developments, supported by an extensive range of statistics. These days, the Monetary Policy Report and Financial Stability Report continue to provide detailed coverage of the data and analytics that have gone into policy formulation. Supervisors now work with a broad range of regulatory returns, with the volume of supervisory data available having increased materially since the global financial crisis.

Nonetheless, the world of data has been changing rapidly and central banks have had to adapt to at least three continuing developments. Global events have presented new policy challenges, most notably the global financial crisis and Covid-19 pandemic. Central banks have often broadened their focus, with some taking on additional macroprudential, microprudential and supervisory roles. More broadly, technological change has led to both vastly more data being available to central banks and the development of powerful new tools to interpret them.

Central banks have had to innovate in response to these developments, although this has not always been easy. They have explored institutional change, including appointing chief data officers, adopting data strategies and experimenting with a range of structures for data governance and management. Many have migrated to cloud. In 2020, 80% of respondents to a BIS survey said that they were using big data sources, up from 30% in 2015.footnote [5] But, at the same time, they have struggled with legacy systems and migrating to new technology, including the unfamiliar IT arrangements that this can involve. New analytics and data practices have needed to fit into existing policy frameworks, including generating reliable results that can be interpreted by policymakers.

The Bank of England has been on the same journey as its major peers. It acquired new responsibilities following the global financial crisis, including: a statutory committee responsible for macroprudential policy; and microprudential policy for, and supervision of, banks, insurers and financial market infrastructures. It has had to adapt to major economic events, including the global financial crisis, the UKs exit from the European Union, the Covid-19 pandemic and, most recently, Russias invasion of Ukraine. And over the past decade or so the Bank has acquired large amounts of new data including microdata on firms and households, regulatory data on banks and insurers and asset or even transaction-level data on key financial products and unconventional data from operational, administrative and digital sources.

The Bank made data a prominent feature of its 2014 One Bank strategy and the strategic priorities for the next three years that it set out in 2021, with both strategies supported by credible assessments of the Banks analytics and data capabilities. In 2014 it created the role of Chief Data Officer, initially at a relatively junior level, before it was made an Executive Director role in 2019. Its data transformation efforts have been supported by an expanding team; from a small Division reporting to the Chief Information Officer it has grown to a full Directorate, bringing together data transformation with Divisions that were already part of the Monetary Policy area, covering advanced analytics and the collation and publication of statistical and regulatory data.

Over the past decade the Bank has taken significant steps to enhance data and analytics. It launched a rationalised and improved suite of analytical tools, which allowed it to focus support and training resources more effectively. It has expanded the range of storage options available, most notably introducing the Data and Analytics Platform to host large data sets. In 2014 it created an Advanced Analytics Division, to act as a centre of excellence. Together, these steps have facilitated increased uptake of programmatic analytical tools and have allowed further centres of excellence to emerge across the organisation.

As a result, the Bank has been able to conduct innovative data and analytics work. Notable examples include: embedding machine learning into the plausibility checking of returns from regulated firms; a predictive analysis tool to support selection of regulated firms for the Prudential Regulation Authoritys (PRAs) Watchlist; tools to analyse insights from firms management information; and, during the pandemic, the rapid adoption of high-frequency indicators from unconventional sources to track economic developments.footnote [6]

The Banks current approach to delivering its mission of promoting the good of the people of the United Kingdom is summarised in its strategic priorities, which support cross-Bank prioritisation. Data appear twice within the current strategic plan, both as a timeless enabler of the Banks mission decision-making informed by the best available data, analysis and intelligence and as Strategic Priority 7, modernise the Banks ways of working. Strategic Priority 7 has two sponsors at Executive Director level, the Chief Data Officer and the Chief Information Officer, as the majority of actions fall to the Data and Analytics Transformation (DAT) and Technology Directorates. The Banks data strategy is an integral part of Strategic Priority 7 and is led by the Chief Data Officer and DAT.

Strategic priorities 202124

The 2021 data strategy had three broad strands. The first focused on enabling, consisting primarily of expanding and refining existing offerings around data collection, storage, support and training. The second consisted of targeted improvements in business outcomes, such as the work in the PRA on RegTech. The third was the Transforming Data Collection programme, run jointly with the Financial Conduct Authority (FCA), which aimed to ensure regulators get the data they need to fulfil their mission, at the lowest possible cost to industry.footnote [7]

DAT, under the ultimate oversight of the Deputy Governor for Monetary Policy and (since 2022) the Chief Operating Officer, is a central function with four roles, all relevant to delivering the data strategy:

DSID plays a key role in several components of the Banks data strategy. They provide key support services, including: management of the Data and Analytics Platform; a Data and Analytics email-based helpdesk; provision of training and guidance to support analytical and data management best practice; and ownership of the Data Catalogue, which is intended to support data governance and act as a repository of key data sources in use across the Bank. DSID also partners with business areas to help them to deliver their priority outcomes through better use of data and analytics. That is supported by the Data and Analytics Business Partners team, which provides a formal link between business areas and experts in the data function, and the Analytics Enablement Hub (AEH), which works with business areas on targeted projects. AEH also provides training and guidance to support the use of a range of modern analytical tools (eg R, Python), strategically selected for the Banks use cases. DSID and the PRA are also working with the FCA and industry to transform data collection from the UK financial sector and have recently established a cross-Bank taskforce to more effectively combine expertise.footnote [8]

More broadly, many of DATs functions are intended to enable business areas to deliver the central data strategy in line with business-area priorities. The management of data assets and many data transformation initiatives sit with individual business areas. For example, after DAT processes and plausibility-checks collections, regulatory data on banks are held and managed by the PRA, while Monetary Analysis manages a database of macroeconomic time series. Business areas across the Bank have begun to experiment with different approaches to strengthening data science skills, whether through training or hiring. Many areas have developed their own centres of analytical excellence specialising in data science techniques. For example, the PRA RegTech team has developed natural language processing tools and acts as a co-ordinating hub for other data specialist teams working in PRA supervision and policymaking areas. Similarly, the Financial Markets Infrastructure Directorates Data team has developed expertise in techniques required to analyse large transaction-level markets data sets. With extensive autonomy, some areas have well-developed data strategies focused on business area priorities in addition to the transforming data collection agenda, the PRAs data strategy covers how regulatory data are accessed, the development of dashboards for supervisors, as well as coaching and digital skills while others have more minimal arrangements. This dispersion of responsibilities was mirrored in oversight, which at the outset of this evaluation was spread across a large number of data or (for investment) programme boards, as well as strategic committees like the Executive Policy Co-ordination Committee, the Executive Operational Co-ordination Committee and the Operations and Investment Committee.

Overall, the current operating model for data and analytics is in transition, partly in response to our evaluation and partly as a result of leadership change.footnote [9] More broadly, important enablers of the Banks data activities are undergoing change: a new plan is being drawn up to tackle technology obsolescence, alongside a cloud migration strategy; central services are being upgraded through the Central Services 2025 (CS2025) programme; and the Banks change capabilities are being strengthened by the appointment of an Executive Director for Change and Planning. Successful execution of these wider initiatives will be crucial to fully delivering the Banks data ambitions.

We took the overarching research question Is decision-making to support the Banks policy objectives informed by the best available data, analysis and intelligence, and can it expect to be so in the future?. We adapted this from the Banks timeless enabler on data, which was set out alongside the 202124 strategic priorities (Figure 1). We broke that down into four evaluation criteria, each underpinned by a set of benchmarks:

We conducted an extensive evidence-gathering exercise, drawing on three main sources:

Launched in 2014, the One Bank strategy was intended as a transformative strategic plan to help the Bank, which had recently expanded to accommodate the newly created Financial Policy Committee and Prudential Regulation Authority, operate successfully as a single organisation.footnote [10] One of the plans four pillars was dedicated to analytic excellence, including making creative use of the best analytical tools and data sources to tackle the most challenging and relevant issues. The strategy saw the creation of the Banks first Chief Data Officer and the Advanced Analytics Division. Specific actions included: external partnering to explore the use of big data and advanced inductive analytics capabilities; and the creation of a One Bank data architecture. The One Bank data architecture aimed to: integrate all the Banks data under the common oversight of the Chief Data Officer; increase the efficiency of data collection and management; share data more widely inside the Bank; and make greater use of third-party providers with economies of scale.

The National Audit Office (NAO) evaluated progress on the One Bank strategy in 2017.footnote [11] It found that of the 15 initiatives planned as part of the strategy, only one was substantively incomplete, the One Bank data architecture. The NAO found that: This turned out to be much more complex than expected, with the Bank identifying that the new IT would need to support around 182 data systems and 2,700 data sets.

Vision 2020, launched in 2017, was the successor strategic plan to the One Bank strategy. Formally, Vision 2020 had a reduced focus on data relative to its predecessor, with data touched on only in the context of data visualisation, under Creative, targeted content, and data sharing, under Unlocking potential. However, in parallel to Vision 2020 in 2017, a data programme was developed as a successor to the One Bank data architecture. Recognising that the previous initiative had been very ambitious relative to the available expertise, budget and planned timescales, the scope agreed in 2017 was narrower and focused on providing self-service tools. Despite the reduced ambition, the 2014 strapline for the programme was preserved: an integrated data infrastructure across the Bank, to enable information sharing. When the programme closed in mid-2020 it was considered to have substantively delivered on the narrower 2017 objectives, though delivery of the Data and Analytics Platform was separated out and was not fully rolled out until 2022.

In 2018, the Bank commissioned Huw van Steenis to write a report on the future of finance.footnote [12] His 2019 report recommended that the PRA embrace digital regulation, including developing a long-term strategy for data and regulatory technology. In its response, the Bank committed to develop a world-class regtech and data strategy.footnote [13] Specific commitments included: consulting supervised firms on how to transform the hosting and use of regulatory data; enhancing the Banks analytics, including peer analysis, machine learning and artificial intelligence; proofs of concept around enhanced analytics and process automation; and making the PRAs rulebook machine readable. As part of delivering its response, the Bank elevated the role of Chief Data Officer to Executive Director level and created the Data and Analytics Transformation Directorate, bringing together a range of existing Divisions.

Launched in 2021, the Banks strategic priorities for 202124 include Strategic Priority 7, modernise the Banks ways of working.footnote [14] This has two elements, one focused on data (described in more detail under current operating model) and one on strengthening the Banks technology. The data elements of the Banks response to the Future of Finance report were incorporated into Strategic Priority 7.

The Bank has consistently set itself a high level of ambition on data and analytics over the past decade and the effective use of data appears prominently in its current strategic plan. The Banks ambitions have been grounded in convincing assessments of the Banks data and analytics capabilities. However, progress has been inconsistent across the organisation with variation in the degree to which ambition has been matched by resources, plans and management oversight. This highlights the challenges inherent in a more devolved data operating model, especially during times of significant transformation in the external data and analytics landscape. Prompted by emerging findings from this evaluation and the arrival of a new Chief Data Officer in April, this area has seen the most change since our evaluation began, which offers a strong foundation for addressing our Theme 1 recommendations around vision, resources, strategy and governance.

Ensuring consensus around the Banks vision for data will be vital, because delivering it will require funding that consistently matches ambition and more concerted championing. The broad support that we heard from the Banks Executive for the current level of ambition suggests the Banks existing timeless enabler could continue to serve as a benchmark for the Banks ambitions. But the renewed strategic conversation currently occurring is needed to restate the case and galvanise support. A renewed vision will need to be met with plans and budgets that consistently match the level of ambition, even if the Bank faces competing priorities. The Bank will also need to review how it tracks spending on data and analytics and make sure funding remains consistent with plans. The Banks senior leaders will need to build on the efforts of the new Chief Data Officer to ensure the centrality of effective data use to the Banks mission is understood both inside and outside the Bank. The PRA has piloted a data skills coaching programme for senior leaders which, if extended Bank-wide, would help support championing efforts.

With an agreed vision and commitment to funding (Recommendation 1), the Banks central functions and business areas will need to work together to refresh the Banks data strategy. This collaborative approach will require common understanding of the art of the possible and of the Bank-wide and individual business areas target operating models, encompassing data, technology and skills. These inputs would support the development of enterprise, data and technology architectures describing their current and target states.footnote [15] As the Bank-wide strategy is refreshed, business areas will need to develop local strategies that are embedded within that and champion them. Nor can the data strategy stand alone it will need to be consistent with, and perhaps developed alongside, supporting strategies for technology and people. In order to build trust, it is important that stakeholders can see measurable progress. That would be aided by: quantified and planned expected benefits ex-ante; mechanisms to track progress; and evaluated outcomes ex-post.

The newly established Data and Analytics Board fills an executive-level gap identified in the early stages of the evaluation and will need to ensure its membership, terms of reference and supporting structures allow the refreshed data strategy (Recommendation 2) to be developed, co-ordinated and monitored during implementation.footnote [16] The Board is a promising development; as it becomes established, its co-chairs the Chief Data Officer and Chief Information Officer and membership will want to ensure that it: remains a forum that effectively convenes central functions and business areas; forges consensus on Bank-wide data and analytics priorities, including the details of the strategy, such as benefits, deliverables and timescales; ensures all the Banks data and analytics transformation activities are consistent with the organisational strategy; keeps abreast of the latest technological developments (Recommendation 7); monitors progress; and holds its members to account for delivery of the strategy and key dependencies. As part of this, it will need to review what supporting structures it requires, including: subcommittees (for example, to ensure data and technology initiatives are consistent with agreed strategies and architectures); monitoring tools (for example, executive scorecards); and accountability devices (such as published documents and member objectives).footnote [17] Our external advisors recommended that governance structures should evolve over time as data maturity increases, suggesting the Bank requires stronger central direction at the early stages of the journey before it can move to a more decentralised approach.

The data analysis produced by Bank staff is highly regarded. Policy committees praise the staffs outputs and its centres of excellence conduct innovative analysis with data. The Bank also continues to rank among the most transparent of central banks. But, not unlike other specialist organisations, the Bank has wrestled in recent years with a range of barriers to making the most of the large amounts of data it acquires. Notwithstanding progress made in recent years, difficulties remain. As with other large, specialist organisations the Bank has found it difficult to combine different types of expertise and to collaborate effectively across business areas, and between business areas and central functions, with local areas preferring to develop their own solutions. A perhaps understandable risk aversion has contributed to: a relatively constrained approach to data sharing, both internally and externally, beyond that necessary due to statutory prohibitions; and a nervousness around adopting new technologies, notably cloud solutions, or working practices. Our recommendations focus on breaking down these institutional, cultural and technical barriers, through: strengthening collaboration, particularly through the use of partner roles linking central functions and local areas; articulating principles to guide greater sharing of data and analytics, internally and externally; strengthening the technological foundations of the Banks data and analytics, particularly by migrating to cloud; and finding ways to draw more extensively on external technical expertise. Continued development of a unified data and technology architecture, supported by improved governance structures, will also be crucial.

The Bank should consider structures that could strengthen collaboration and more effectively combine expertise. This applies across business areas, between business areas and central functions, and between different professions (particularly data, change management and technology specialists). Its business partnerships programme if fully implemented offers a promising start, focused on building collaboration between business areas and the data function. This could play a crucial role in helping: central functions understand desired business outcomes; business areas understand what is possible; and the Bank in ensuring that data projects can be incorporated within Bank-wide data and technology architectures. The Bank will need to monitor progress on business partnerships, including a balanced assessment of how business areas have engaged with it, perhaps at the Data and Analytics Board. Further action may be needed to reinforce cultural change. The Bank should review lessons from the partnerships and the newly established cross-Bank AI and data collection taskforces when considering the most effective ways to bring together people with common interests and expertise. More broadly, we came across interesting models at peer central banks, including those focused on combining data and technology experts with business area specialists to produce repeatable products. There are also a range of models (eg guilds, tribes) and delivery frameworks (eg the Data Management Capability Assessment Model) established in the data management profession for combining expertise.footnote [18] This will have wider implications, since CS2025 also proposes partnership models for the Banks Technology and People Directorates.

Greater openness around data and analytics, internally and externally, would foster greater scrutiny and challenge, helping the Bank gain additional insights and keep up with a rapidly evolving world the Bank produces large amounts of extremely valuable data and analysis, but much of it is only easily available to subsets of Bank staff. The Bank could adopt a presumption of sharing, but would need to further consider the implications and appropriate guardrails. The Bank has important legal obligations and constraints when it comes to sharing data but, within those, it should articulate and highlight a set of principles for disseminating data and analytics, internally and externally. Guiding principles would allow the Bank to consider how to safely open up wider access to data and analysis and might facilitate external collaboration. This is consistent with the IEOs Research Evaluation, which recommended that the Bank needed to support access to data for external co-authors to broaden the expertise and perspectives that the Bank can draw on. We note that some other organisations that face similar binding restrictions have found means to facilitate access to internal data, for example the ONS Secure Research Service.footnote [19] Moving to cloud (Recommendation 6) could help overcome technical barriers to sharing.

The Bank will need to develop an achievable plan to modernise most of its data and analytics practices, to avoid falling further behind a rapidly evolving frontier. A reliance on inefficient manual processes generates risks and staff frustration. A move to cloud would be the most powerful technological step the Bank could take to close this gap, supported by common standards (Recommendation 8) and upskilling (Recommendation 10). While a move to cloud is no panacea and brings new challenges, we have seen that peers and other organisations have been able to unlock capabilities through the use of modern tools and provide increased computing capacity. Cloud offers a range of enhanced capabilities that could improve data collection, discoverability (for example, automation of data cataloguing) and analysis, and allow some embedding of modern data management practices (Recommendation 8). This could include being more open to buying in off the shelf tools than is currently the case. Peers experience suggests cloud migration might also help with other issues such as efficient use of licenses, facilitating access for external experts (Recommendation 5) and obsolescence, with cloud providers keeping tools up to date. As a late adopter of cloud, the Bank can learn lessons from other organisations experience of making the transition.

The Bank will need to consider the role of its emerging centres of excellence in raising data maturity; centres of excellence like Advanced Analytics and local business area data science hubs have brought deep expertise into the Bank and, through collaboration, have helped develop others skills. The Bank should ask whether there is more it can learn from others experience of innovation hubs and how their role should evolve as maturity rises. Further mechanisms might include an external advisory board made up of experienced experts to provide challenge to the Data and Analytics Board; the Bank has used such arrangements effectively in a number of areas and the Monetary Authority of Singapore have used it for data and technology.footnote [20] footnote [21] Coaching senior staff on data could be expanded Bank-wide (Recommendation 1) and draw on lessons from elsewhere, for example reverse-mentoring, where talented analysts and data specialists coach senior staff on the art of the possible.

The Bank has long understood the importance of enabling individuals to work effectively with data. The 2014 data strategy focused on enabling business areas and encouraging individuals to get and make better use of data within their roles is a key outcome in the Banks 2021 data strategy, with DAT identifying data, tools, platforms, training and other support services as important enablers to facilitate that outcome. While the Bank has introduced data specialist roles and built up its training offering, developing data skills and embedding new, more modern approaches takes time. Recent technological developments have increased analytical capabilities and offer the potential to automate more processes, freeing staff time to focus more on higher value-added analysis. This may have implications for the skills the Bank wants to develop and how staff best work with each other. We have identified three recommendations to help the Bank make progress by ensuring staff have the support and skills they need: embed common standards to make data and analysis easily discoverable and repeatable; provide staff with accessible support and guidance across the data lifecycle; and develop a comprehensive data skills strategy encompassing hiring, training, retention and skills mix.

The Banks data and analytics guidance needs to be comprehensive, easier to find and better incentivised. The Bank has recently refreshed its guidance on data management, which will be launched this year. It would benefit from doing the same for analytical common standards, not least to ensure that greater use of programmatic analytical tools is suitably resilient. When these are established, the Bank should consider how to raise engagement and adherence. Training is one option, and is standard at induction in professional services firms, consultancies and banks. Other options include greater leadership championing, recognition of good practice in performance reviews, mandatory training, audits and individuals attesting to compliance with the standards through the Our Code process.footnote [22] In the past, the Bank has used the performance management process to influence behaviour, or enforced compliance top-down. Best practice could be built into future data and analytics platforms as part of cloud migration (Recommendation 6), which could include automation of data cataloguing which our advisors indicated was widely adopted in data-mature private sector firms.

The Bank could go further in joining up existing data and analytics support. This would materially enhance the support accessible to staff, who currently struggle to navigate the fragmented range of services available. A single front door, clearly visible on the desktop or intranet front page and spanning the data lifecycle, could effectively triage data and analytics requests, directing them to existing resources or escalating to deeper support, as appropriate. The Bank already operates elements of this approach in interactions between hubs, helpdesks and users joining it up would significantly ease the experience of staff. The Bank should define how this service would relate to that offered by business partners and how those business partners are resourced to meet any increased demand.

The Bank should develop a comprehensive data skills strategy, embedded within wider initiatives around talent and skills. Such a strategy will need to consider the career proposition for data specialists, including both data scientists and the technology specialists that support data work. It will need to articulate where skills should be developed inside the Bank, across all levels of seniority and supported by a training offer, or hired into the organisation. It will need to be informed by a clearly defined operating model (Recommendation 2) that articulates the role of data specialists relative to other skillsets in the organisation, including analysts and technology specialists. This should be linked to the People Directorates wider talent strategy. The Bank can draw lessons from the mix of approaches business areas have adopted when experimenting with building area-wide data skills.

Read more here:

IEO evaluation of the Bank of England's use of data to support its ... - Bank of England

Read More..