The pandemic accelerated the urgency for reform in health and social care around the world, which strained resources to unprecedented levels. The effects are still being felt and in Northern Ireland specifically, ongoing political instability is further complicating approaches to digital transformation. Although progress is being made that should be recognized and celebrated, Dan West, CDIO for Health and Social Care in Northern Ireland’s Department of Health, understands that the pandemic still casts a lingering shadow over national health and care systems, contributing to continuing rampant fatigue among staff and subsequent strikes over pay.

“From a people perspective, things are pretty strained at the moment,” he says. “All of the capacity and operational challenges that were present in healthcare prior to the pandemic are magnified now. Waiting lists have grown and diagnosis and treatments have been delayed or missed due to some of the burden the pandemic brought on the system. Plus, the absence of a functioning devolved executive, due to an ongoing dispute by the DUP over the Northern Ireland protocol, adds to all of those challenges. It all reduces the ability to arrive at a budget settlement that shifts resources into the health and social care space. So you can see how my job has had to react to all of those stimulants.”

Leaders in the public sector and healthcare might worry about a return of red tape that could slow down innovation, too, but West has a more resilient outlook to progress by, as he says, never wasting a crisis.

“We’ve been able to accelerate the things we knew we needed, but we also had a rate of adoption of collaboration and flexible working tools that we wouldn’t have seen in peacetime, if you like,” he says. “I was doing some work in a trust in the English NHS and we gave everybody tools, laptops and mobile solutions, and redesigned the operating model and how we worked together. There was resistance, though, where people felt they still needed an office, but there was an exponential increase in the use of virtual tools and capabilities to how we interacted with each other as professionals, and with our patients. We need to make sure that how we deliver digital capability to our staff, and how we do digital enablement of services for citizens, is not allow that elasticity in bureaucracy to snap us back to traditional ways of working.”

CIO Leadership Live’s Drinkwater recently spoke with West about how to put people first in a system under increasing pressure to function as it strives to digitally transform amid a backdrop of political and environmental uncertainty. Watch the full video below for more insights.

On balancing efficiencies: To sustain health and social care services into the future, we need to find a way to get more output from the same or maybe even reducing resources. And I don’t think anybody would suggest that digital is some kind of panacea to all of this. The real requirement is in and around staff. But the absence of the money to hire more doctors and nurses, the lead time, and then training them to bring them into the service means that digital has to be part of the jigsaw puzzle to address those challenges. The projects and products we delivered during COVID-19 adopted some of the techniques and technologies that allowed more efficient digitally enabled services.  

On continuing important work: There’s an impact on the experience of citizens, interacting with health and social care services, and leveraging cloud technologies, smart phone apps, and agile delivery methodologies that allowed us to quickly put things in place in a way we haven’t done previously. The COVID Care NI app, for instance, is like a symptom checker and chat bot that provides a personalized package of advice based on personal circumstances and how it related to evolving regulations. Also, the Bluetooth proximity app Stop COVID NI was the first of its kind to launch in the UK, and the first globally to achieve international interoperability, given our shared border with the Republic of Ireland and the Epidemiological Unit of the Island of Ireland. It was important for us to create our sharing capability. A new way of delivering vaccines and vaccine management and convenient booking capabilities was also big. All of those things really changed the paradigm. It moved away from the traditional, matriarchal approach to delivering healthcare services, galvanizing and empowering patients and their families more and how they interacted with services. I’m interested in how we could maintain momentum around those experiences in a post-pandemic world, to leverage technologies and techniques like that, to deliver improved and modernized experiences and services.

On progressing the digital conversation: The fail fast mentality to get product out there that can help us change the way people work and live is one of the key focuses for all technologists in parallel to public scrutiny. People will be more risk-averse than they were during the pandemic, so let’s not allow those big public processes to push us back into traditional ways of working. What we saw was the digital tail wagging the clinical dog. Historically, we tried to avoid that. We tried to have the operational care model and service design drive out technology requirements. I’m not suggesting that shouldn’t be how the world works, but during COVID, the technologists could go beyond the current operating environment where we were trying to build the pandemic response, and look around the world and see the pockets of innovation and best practice—like building a new vaccine management platform with a publicly accessible appointment booking service. That didn’t come from an operational or clinical policy discussion. It was something a colleague in Scotland introduced me to, and I then took it as a digital reader to the Public Health Operational and Clinical Response to say, “We need to start thinking about this.” We can bring digital upstream in the policy conversation and collaborate differently with clinicians who have a better understanding of the benefit and potential impact that digital can have, as well as the way they form relationships and transactions with citizens. All those considerations that allow us to alter the dynamic and accelerate how we can transform services is an environment that’s resistant to change.

On grassroots involvement: I had never done startup stage work in technology, but it’s something I’m interested in to build my own experiences. I’m also quite passionate about the local technology sector. So I’m helping a young reg-tech and insure-tech business to create an initial suite of products that will work within governments, the insurance industry, and home and business owners to address the fact that one in five properties in the UK can’t secure risk-reflective and affordable insurance against flood damage. That’s not sustainable for us, as a society. So technology has a part to play in the information relationship between governments, the insurance industry, and those businesses and homeowners, so that as we increase resilience of our properties through flood risk interventions, we can build that body of evidence that drives us toward a risk-reflected market in insurance. It’s a really interesting opportunity and a different environment to work in compared to the health and care day job. Belfast is getting a good reputation now for technology, so getting involved in a startup and taking it through a launch was quite an interesting opportunity.

On the value of people: To continue delivering and working through priorities for our trusts, and make the digital journey achievable is we have to recognize that projects and programs are just a means to an end. The technology itself is not what we’re all about. It’s about the people being able to deliver better healthcare services, our staff being able to do their jobs better, and our citizens interacting with safer, modern and more convenient healthcare experiences. It’s the opportunity to shift health and social care from being an economic burden for society in Northern Ireland to a real opportunity for growth and innovation. And those people aspects of that technology portfolio must be our focus over the next few years as we carry on this digitization journey in health and care. In a resource constrained environment, we have to evidence that all of these technology investments are delivering a real change for people in the region.

CIO, Digital Transformation, Healthcare Industry, IT Leadership

Digital transformation has reached a critical juncture within the railway industry. As rail operators embrace new trends in intelligence, sustainability and service, aging telecommunications architecture of more than 20 years ago is unable to meet current and future requirements.

The existing GSM-R train-to-ground communication system can no longer provide sufficient capacity for modern railway stations. Instead, operators are turning to the Future Railway Mobile Communication System (FRMCS) with broad bandwidth and a new decoupling architecture based on LTE and the latest technology to improve performance.

“Digital transformation is a long journey and rail operators need to ‘dream big’. Currently, the most pressing challenge for rail operators is to identify both the pain points and benefits and address them with cost-effective digital solutions—to ‘act small,’” said Xiang Xi, Vice President, Aviation & Rail BU, Huawei Technologies Co., Ltd.

As an industry-leading ICT solution provider, Huawei can help with these transformation efforts in three aspects: by reshaping connectivity, reconstructing the platform, and enabling intelligence. At the upcoming InnoTrans exhibition, Huawei will outline the framework for digitalisation of the railway business and best practices for innovation and showcase smart railway related Solution.

Reshape connectivity

With digitalisation, demand for new services such as train automation, smart maintenance, and others is growing. Current narrowband network has insufficient bandwidth to meet complex network requirements to support these services. Innovative solutions such as FRMCS, Wi-Fi 6 and all-optical networks will enable a more digitalised rail infrastructure.

Huawei FRMCS solution enables wireless communications systems for high throughput, low latency, and reliable connectivity. This solution can support new railway services such as multimedia dispatching communications, trackside IoT, and predictive maintenance.

Reliability in connectivity also requires zero interruptions for real-time service. Wi-Fi 6 Train-to-Ground communication, Railway All-Optical Network using native hard pipeline (NHP) and Urban Rail Cloud-Optical Network based on OTN technology with ultra-low latency, enables no-disruption connectivity to operators’ assets, services, operations and maintenance.

Reconstruct platform

Traditional urban rail lines and service systems, including ATS, AFC, and PIS, are relatively independent. The silo construction of IT resources leads to high construction costs, low resource utilization, and isolation of multiple information systems. The unified construction mode of the urban rail cloud changes these issues.

Huawei’s Urban Rail Cloud Platform solution empowers rail operators to maximise IT resources and improve security and efficiency of operations.

Enable intelligence

As rail operations increase in complexity, rail operators need to gain a better situational awareness in order to manage their assets better and expedite incident response time. The challenges becomes even greater as operators look for ways to improve low-carbon development and other parts of their operations to address sustainability objectives.

Huawei’s Urban Rail Intelligent Operation Center (IOC) solution connects digital environments with physical spaces for improved and integrated situational awareness leading to better decision-making and an efficient and collaborative command.

Next-generation communications technologies will play a critical role in addressing the unique challenges of the rail industry. Cutting-edge solutions such as FRMCS, Wi-Fi 6, and all-optical networks are addressing those challenges to empower the rail industry to modernize for improved safety and reliability of rail lines, while opening new opportunities for innovation.

Register now to find out Huawei’s global experience in the rail industry at the 9th Huawei Global Rail Summit on 22nd September at the Grand Hyatt Berlin in Germany.

Digital Transformation

Since 2015, the Cloudera DataFlow team has been helping the largest enterprise organizations in the world adopt Apache NiFi as their enterprise standard data movement tool. Over the last few years, we have had a front-row seat in our customers’ hybrid cloud journey as they expand their data estate across the edge, on-premise, and multiple cloud providers. This unique perspective of helping customers move data as they traverse the hybrid cloud path has afforded Cloudera a clear line of sight to the critical requirements that are emerging as customers adopt a modern hybrid data stack. 

One of the critical requirements that has materialized is the need for companies to take control of their data flows from origination through all points of consumption both on-premise and in the cloud in a simple, secure, universal, scalable, and cost-effective way. This need has generated a market opportunity for a universal data distribution service.

Over the last two years, the Cloudera DataFlow team has been hard at work building Cloudera DataFlow for the Public Cloud (CDF-PC). CDF-PC is a cloud native universal data distribution service powered by Apache NiFi on Kubernetes, ​​allowing developers to connect to any data source anywhere with any structure, process it, and deliver to any destination.

This blog aims to answer two questions:

What is a universal data distribution service?Why does every organization need it when using a modern data stack?

In a recent customer workshop with a large retail data science media company, one of the attendees, an engineering leader, made the following observation:

“Everytime I go to your competitor website, they only care about their system. How to onboard data into their system? I don’t care about their system. I want integration between all my systems. Each system is just one of many that I’m using. That’s why we love that Cloudera uses NiFi and the way it integrates between all systems. It’s one tool looking out for the community and we really appreciate that.”

The above sentiment has been a recurring theme from many of the enterprise organizations the Cloudera DataFlow team has worked with, especially those who are adopting a modern data stack in the cloud. 

What is the modern data stack? Some of the more popular viral blogs and LinkedIn posts describe it as the following:

Ben Patterson/IDG

A few observations on the modern stack diagram:

Note the number of different boxes that are present. In the modern data stack, there is a diverse set of destinations where data needs to be delivered. This presents a unique set of challenges.The newer “extract/load” tools seem to focus primarily on cloud data sources with schemas. However, based on the 2000+ enterprise customers that Cloudera works with, more than half the data they need to source from is born outside the cloud (on-prem, edge, etc.) and don’t necessarily have schemas.Numerous “extract/load” tools need to be used to move data across the ecosystem of cloud services. 

We’ll drill into these points further.  

Companies have not treated the collection and distribution of data as a first-class problem

Over the last decade, we have often heard about the proliferation of data creating sources (mobile applications, laptops, sensors, enterprise apps) in heterogeneous environments (cloud, on-prem, edge) resulting in the exponential growth of data being created. What is less frequently mentioned is that during this same time we have also seen a rapid increase of cloud services where data needs to be delivered (data lakes, lakehouses, cloud warehouses, cloud streaming systems, cloud business processes, etc.). Use cases demand that data no longer be distributed to just a data warehouse or subset of data sources, but to a diverse set of hybrid services across cloud providers and on-prem.  

Companies have not treated the collection, distribution, and tracking of data throughout their data estate as a first-class problem requiring a first-class solution. Instead they built or purchased tools for data collection that are confined with a class of sources and destinations. If you take into account the first observation above—that customer source systems are never just limited to cloud structured sources—the problem is further compounded as described in the below diagram:

Unisys

The need for a universal data distribution service

As cloud services continue to proliferate, the current approach of using multiple point solutions becomes intractable. 

A large oil and gas company, who needed to move streaming cyber logs from over 100,000 edge devices to multiple cloud services including Splunk, Microsoft Sentinel, Snowflake, and a data lake, described this need perfectly:

Controlling the data distribution is critical to providing the freedom and flexibility to deliver the data to different services.”

Every organization on the hybrid cloud journey needs the ability to take control of their data flows from origination through all points of consumption. As I stated in the start of the blog, this need has generated a market opportunity for a universal data distribution service.

P Wei / Getty Images

What are the key capabilities that a data distribution service has to have?

Universal Data Connectivity and Application Accessibility: In other words, the service needs to support ingestion in a hybrid world, connecting to any data source anywhere in any cloud with any structure. Hybrid also means supporting ingestion from any data source born outside of the cloud and enabling these applications to easily send data to the distribution service.Universal Indiscriminate Data Delivery: The service should not discriminate where it distributes data, supporting delivery to any destination including data lakes, lakehouses, data meshes, and cloud services.Universal Data Movement Use Cases with Streaming as First-Class Citizen: The service needs to address the entire diversity of data movement use cases: continuous/streaming, batch, event-driven, edge, and microservices. Within this spectrum of use cases, streaming has to be treated as a first-class citizen with the service able to turn any data source into streaming mode and support streaming scale, reinforcing hundreds of thousands of data-generating clients.Universal Developer Accessibility: Data distribution is a data integration problem and all the complexities that come with it. Dumbed down connector wizard–based solutions cannot address the common data integration challenges (e.g: bridging protocols, data formats, routing, filtering, error handling, retries). At the same time, today’s developers demand low-code tooling with extensibility to build these data distribution pipelines.

Cloudera DataFlow for the Public Cloud, a universal data distribution service powered by Apache NiFi

Cloudera DataFlow for the Public Cloud (CDF-PC), a cloud native universal data distribution service powered by Apache NiFi, was built to solve the data collection and distribution problem with the four key capabilities: connectivity and application accessibility, indiscriminate data delivery, streaming data pipelines as a first class citizen, and developer accessibility. 

IDG

CDF-PC offers a flow-based low-code development paradigm that provides the best impedance match with how developers design, develop, and test data distribution pipelines. With over 400+ connectors and processors across the ecosystem of hybrid cloud services including data lakes, lakehouses, cloud warehouses, and sources born outside the cloud, CDF-PC provides indiscriminate data distribution. These data distribution flows can then be version controlled into a catalog where operators can self-serve deployments to different runtimes including cloud providers’ kubernetes services or function services (FaaS). 

Organizations use CDF-PC for diverse data distribution use cases ranging from cyber security analytics and SIEM optimization via streaming data collection from hundreds of thousands of edge devices, to self-service analytics workspace provisioning and hydrating data into lakehouses (e.g: Databricks, Dremio), to ingesting data into cloud providers’ data lakes backed by their cloud object storage (AWS, Azure, Google Cloud) and cloud warehouses (Snowflake, Redshift, Google BigQuery).

In subsequent blogs, we’ll deep dive into some of these use cases and discuss how they are implemented using CDF-PC. 

Get Started Today

Wherever you are on your hybrid cloud journey, a first class data distribution service is critical for successfully adopting a modern hybrid data stack. Cloudera DataFlow for the Public Cloud (CDF-PC) provides a universal, hybrid, and streaming first data distribution service that enables customers to gain control of their data flows. 

Take our interactive product tour to get an impression of CDF-PC in action or sign up for a free trial.

Data Center