---
title: Fusion Theme Blog | Data Architecture & Integration
description: Data Architecture & Integration | Fusion theme blog
---

# Articles about Data Architecture & Integration

Show all categories  AI & Machine Learning  Application Development  Application Modernization  BI & Analytics  Cloud  Cloud Strategy & Transformation  Customer Data Strategy  Customer Experience Strategy  Data  Data Architecture & Integration  Data Virtualization  Digital  Digital Analytics  Digital Experience  Digital Marketing  Digital Transformation  Emerging Technologies  Experience Design  Infrastructure Services  Modern Data Platforms  Page: AI & ML  Page: Cloud Infrastructure  Page: Cloud Strategy  Page: CX Consulting  Page: Data architecture & integration  Page: Mobile App Development  Page: Strategic data mangement  Strategic Data Management  Technology  Technology Strategy

[![](https://22034068.hs-sites.com/hubfs/blog-features/data-virtualization-imperatives.jpg) ](https://22034068.hs-sites.com/fusion-blog/articles/modern-data-platform-imperatives)

## [5 Imperatives of a modern data platform](https://22034068.hs-sites.com/fusion-blog/articles/modern-data-platform-imperatives)

Nov 21, 2022

 Accessing data at the speed of business is critical to remaining competitive in a digital-first world. But if you’re relying on outdated architecture where your data is trapped in silos or lost in a data lake, access to the functional data you need is seriously limited. When your existing framework is no longer serving your business, it makes sense to transition to a modern data platform, but you may have hesitations about whether it can help you succeed. To help you better understand this solution and what you need to gain from it, we are looking at data platform capabilities and sharing five modern data platform imperatives that will help achieve a more logical data management system. What is a modern data platform? With so many emerging data solutions, we understand that data is a very complicated environment, so we want to start by clearly defining what a modern data platform is and its capabilities. A modern data platform is a flexible, cloud-based, end-to-end data architecture that supports collecting, processing, analyzing, and delivering data to the end user in a way that is aligned and responsive to the needs of the business. On the surface, aside from it being cloud-based rather than on-premise, modern data platform capabilities aren’t different from traditional data architecture. The difference is in how new technologies have expanded their capabilities. Here are some of the ways modern data platforms can deliver more for your organization: Data ingestion Bringing new data into the environment is the first step to managing data, and in a legacy architecture, that is mainly done through batch processing. Batching collects and processes data at specific time periods or intervals. By leveraging the higher computing capacity of a cloud-based architecture, data can be streamed in real time to data storage units, eliminating bottlenecks and delays to keep data moving through the system in a more fluid manner. Quality and governance With AI integrated into the architecture, data quality and governance tools can be automated, speeding up how new data sources are analyzed, categorized, and assessed for security concerns. Security Security measures can be integrated at the base level for new data products, providing inherent encryption whether it’s at rest or in transit. Within a modern data platform, security measures are implemented to dynamically filter and obscure data as needed to support your organization’s security policies. Storage Cloud-based architecture offers the potential for nearly unlimited storage and offers a pay-as-you-go model, so you only need to invest in the volume of storage you need today. As your data storage needs increase in the future, you can add and seamlessly integrate additional space without creating silos for new data. Transformation In legacy architecture, transformations such as quality adjustments and business logic need to be applied in the early stages of data flow during large batch processing. While this ensures that the downstream usage of the data is more performant, it also locks the business rules in place which removes flexibility in how the business looks at and interacts with the data. The expanded computing power and advanced tools in a modern data platform offer a more flexible timeline to add transformations to the data. Business rules and logic can be applied later in the data flow and adapted to suit changing needs. Discovery Data discovery is streamlined through integrated tools within a modern data platform that can automatically scan, categorize metadata, and organize it so the most appropriate data is accessed more easily and quickly. Delivery In a legacy architecture, data delivery visualization tools required the data to be specifically structured prior to business usage, whether for reporting, data extracts, or API access. Now, visualization tools have advanced features that support access to semi-structured and unstructured data without the need for intensive (and expensive) data processing. Integrated tools simplify both data extraction and data sharing and have built-in security and monetization features. DevOps and DataOps In a modern data platform, DevOps/DataOps are cross-platform and cross-language supportive, which makes it easier and faster to coordinate development and release implementation tasks when architectures are built using multiple tools. 5 modern data platform imperatives The overall framework, capabilities, and patterns of managing data are universal within a modern data platform. However, no two platforms are the same. Each one is highly customized to support the data and data needs of the organization and require different combinations of tools or features to achieve specific functionalities and cover the needed capabilities. You still need to ensure your platform manages the data in a way that aligns to your organization’s unique needs, and this means that five modern data platform imperatives must be met. 1. Greater flexibility The greatest challenge of legacy data architecture is the lack of flexibility. The physical servers can’t be added to or modified easily to meet the changing data needs of your organization, so they need to be built with the capacity for future data needs. This is easier said than done given the rapidly changing landscape and the sheer volume of data you’re taking in. A modern data platform is incredibly flexible. It allows you to consider your data needs today and budget accordingly rather than trying to predict your data needs in the future which requires a significantly larger investment. As you need to increase data storage, adopt automation, or pivot in your data needs, these updates can be integrated seamlessly into the platform. 2. Improved access The people and applications accessing data need it in real time and in the proper format, but the needs of your data science team vary greatly from the needs of your business intelligence team. A modern data platform must support a faster time to market for data assets, and one way it does this is through a medallion architecture. A medallion architecture creates a multi-layered framework within the platform to move data through a pipeline to the end user. Bronze layer: Raw data is collected directly from the source systems with little to no transformation and stored here to provide a base layer of full history for additional processing. Silver layer: Data from multiple sources are curated, enriched, integrated, and organized in a structure that reflects the data domains of the organization. Gold layer: Data needed to support specific business drivers is aggregated and organized so it can be used for dashboard creation and self-service analysis of current states and trends. This architecture allows a diverse user base to access the data in the form that best suits their needs. Data scientists can access raw data from the bronze layer to identify new and emerging patterns, business applications can access data in the silver layer to produce data products, and business users can access the gold layer to perform analytics and create dashboards. 3. Incremental implementation Rather than transitioning to a modern data platform in a single, giant step, we recommend an incremental move. This makes it significantly easier and faster to focus on the current data products your organization needs, like reports and dashboards, while you are starting to build out the initial infrastructure. An incremental implementation lets you take a clear, informed look at the data you need, how you need it, and how it aligns with your business drivers. You can then choose to add, adjust, or stop processing certain data to put more focus on the data that will answer pivotal business questions. At the same time, building only what you need when it’s needed, an incremental implementation saves money and avoids bringing over old data that no longer serves your business. 4. Better communication between IT and business users A modern data platform needs to support improved communication between your IT or data engineers and your business users. As data flows through the framework and reaches the end user in the language they speak, the end-user has greater clarity. For business users, this may mean seeing gaps in how the existing data is not directly answering their questions and needs to find a different way to utilize the data. For the data engineers, this may mean seeing opportunities in how to filter out aberrations in the data to improve the aggregated data. This clarity allows the teams to work together to target solutions that will cover existing or emerging needs. 5. Re-focus valuable resources Once the initial data set is built, we apply repeatable patterns to the mechanics controlling data ingestion, storage, and delivery. Having a proven framework that can be repeated to unlimited data sets saves time and reduces the cost of building, operating, and maintaining the platform. Your data team can refocus their time on higher-level tasks, including improving data quality and speeding up delivery. Whether you have questions about data platform capabilities and functionalities or you’re ready to make the shift to a modern data platform, we’re here to help! Set up a call to talk to an expert or visit our modern data platform hub to learn more. Ask us your questions >> Learn more about modern data platforms >>

[![](https://22034068.hs-sites.com/hubfs/blog-features/data-fabric-vs-data-mesh.jpg) ](https://22034068.hs-sites.com/fusion-blog/articles/data-fabric-vs-data-mesh)

## [Data fabric vs data mesh: Choosing the best data architecture for your organization](https://22034068.hs-sites.com/fusion-blog/articles/data-fabric-vs-data-mesh)

Nov 14, 2022

 Whether your data is housed in a monolithic data architecture or across multiple, disparate sources such as databases, cloud platforms, and business applications, accessing the specific information you need when you need it probably presents a huge challenge. The length of time it takes to find data may have you or your analytics teams constantly relying on outdated information to run reports, develop strategies, and make decisions for your organization. If you’re exploring data solutions that will improve time-to-market while simplifying governance and increasing security, you’ve probably come across the terms “data fabric” and “data mesh,” but you may not know how to apply them to your business. To help you better understand these emerging trends in data architecture, we’re digging into what a data fabric and data mesh are and the specific benefits they bring to large and enterprise-level organizations. This will give you the foundational knowledge to determine how to choose data fabric vs data mesh or how both may be able to serve your organization. What is data fabric? When you think of every bit of data in your organization as an individual thread, it makes sense that it takes so long to access specific information. If thousands of individual threads are stored together in a bin, like in a monolithic architecture, or separated across hundreds of individual boxes with little to no organizational method, like in a distributed architecture, how long would it take to find the single thread you’re looking for and get it untangled so you can use it? A logical data fabric solves this problem by weaving all the threads of data together into an integrated, holistic layer that sits above the disparate sources in an end-to-end solution. Within the layer are multiple technologies working together to catalog and organize the data while machine learning and artificial intelligence are implemented to improve how new and existing data are integrated into the fabric as well as how data consumers access it. Are data virtualization and data fabric the same? A common misconception is that data virtualization and data fabric are the same. On the surface, they both support data management through the creation of a single, integrated layer of processed data atop distributed or unstructured data. Data virtualization is an integrated abstraction layer that speeds up access to data and provides real-time data returns, and this technology is a key component within the data fabric. However, data virtualization is still only one of the multiple technologies comprising the entity, which is a more comprehensive data management architecture. Benefits of data fabric Now that you have a better understanding of what data fabric is, let’s consider the problems it solves and why it may be right for your organization. Access your data faster When your data is in multiple formats and housed in a variety of locations, gaining access to the specific details you need can take hours, days, or even weeks, depending on your architecture. A logical data fabric leverages metadata, semantics, and machine learning to quickly return the needed data from across multiple sources, whether it’s a large amount of historic information or highly specific data used to drill down into a report. Democratize your data Data fabric uses advanced semantics, so the data is accessible in the language of business users, such as BI and analytics teams. Data consumers within the organization can access what they need without having to go through data engineers or the IT department, eliminating bottlenecks and sharing ownership of data. Improve governance Because of the automation capabilities of data fabric, you can implement a governance layer within the fabric. This applies global policies and regulations to data while allowing local metadata management to reduce risk and ensure compliance. What is data mesh? Monolithic data architecture keeps data in one centralized location. On paper, this seems like a more cost-effective, efficient option compared to a distributed architecture, but it still brings several challenges. Consider that in many large organizations relying on a monolithic architecture, massive volumes of unstructured data are stored in a data lake. For information to get into the hands of data consumers or before productization can occur, the data must be accessed and processed through the IT department, creating significant bottlenecks and bringing time to market to a crawl. A data mesh can solve this challenge. This is a new type of data architecture, only proposed in 2019 by Zhamak Dehghani of Thoughtworks, in which a framework shifts data from a monolithic architecture to a decentralized architecture. More specifically, the data is distributed across autonomous business domains where the data consumers own, manage, and share their own data where they see fit. While the domains are given a separate virtual schema and server so they can have full ownership over data productization, governance, security, and compliance are still unified within the monolith. Benefits of data mesh The challenges of centralized data ownership include latency, added costs of storage, software, replication, and lack of practical access for consumers, but implementing a data mesh can solve these. Eliminate IT bottlenecks When all data is forced to go through the IT department before being distributed to the individuals or teams requesting it, bottlenecks occur and slow down the flow of data. A data mesh allows data to bypass the IT department, allowing data to flow freely to the needed source. Improve flexibility and agility Finding specific information within the massive volume of unstructured, undefined data stored in a data lake requires increasingly complicated queries to get the needed information. However, a data mesh gives ownership of datasets to individual teams or business owners, simplifying access and offering real-time results through scalable, automated analytics. Increase connection to data By transferring data ownership to the data consumers, those who use it directly have a greater connection to it. The data is available in the language of business, and it can be shared across teams with greater ease and transparency. Choosing data fabric vs data mesh Data fabric and data mesh both support data democratization, improve access, eliminate bottlenecks, and simplify governance. While data fabric is built on a technology-agnostic framework to connect data across multiple sources, data mesh is an API-driven, organizational framework that puts data ownership back in the hands of specific domains. So, which is better in the debate between data fabric vs data mesh? The simple answer is neither one is better than the other, and the right option is determined by the use case. If the goal of your organization is to streamline data and metadata to improve connection and get real-time results across multiple teams, a data fabric built on a data virtualization platform can help you meet your goals. On the other hand, if you need to improve the process of data productization and decentralizing your data, a data mesh may be the best option. But the real answer is that contrary to popular belief, the two are not mutually exclusive and most businesses succeed by implementing both options. Data fabric and data mesh are complementary solutions that can work together to solve the challenges of your existing architecture. Learn more about data fabric and data mesh Want to gain further insight into choosing data fabric or data mesh? We partnered with data management leader Denodo Technologies for a recorded webinar. In Logical Data Fabric vs Data Mesh: Does It Matter? we provide an in-depth look at monolithic and distributed data architecture, the challenges they bring, and how both data fabric and data mesh can improve agility, reduce costs, and elevate the quality of your data. To ask additional questions or learn how Fusion Alliance can help you create and implement a successful data strategy to meet your unique challenges and goals, connect with our team today. Learn more about modern data platforms >>

[![](https://22034068.hs-sites.com/hubfs/blog-features/Is%20your%20data%20cereal%20or%20soup.jpg) ](https://22034068.hs-sites.com/fusion-blog/articles/is-your-data-cereal-or-soup)

## [Is your data cereal or soup?](https://22034068.hs-sites.com/fusion-blog/articles/is-your-data-cereal-or-soup)

Oct 12, 2022

 The importance of data classification Often presented as a click-bait internet poll, the question “Is cereal a soup?” is only baffling until you realize that the answer hinges on how you define the term. Merriam-Webster contends that soup is a liquid sustenance often containing pieces of solid food. Therefore, as one respondent said, cereal is a soup “technically, though not existentially.” Proper definition of terms is also critical when it comes to classifying your data. To get the most from your data assets, you’ll need a strong data strategy, supported by definitions like: How information is grouped, weighted, and prioritized How common dimensions will be conformed How data will be standardized, cleansed, and tagged Your data use cases, sources, and architecture are unique. How you define your data strategy should be, too. Fusion’s team of data, technology, and digital experts can help you architect and implement a comprehensive data strategy, offer insights and best practices to support a growing data culture, or step in to solve a particular problem. Don’t let data eat your business for breakfast. Learn more about defining your data terms or get in touch for a quick consultation. This article was originally shared in our Fuse newsletter — get content like this directly in your inbox. Subscribe to Fuse

[![](https://22034068.hs-sites.com/hubfs/fusion/speed-to-value.jpg) ](https://22034068.hs-sites.com/fusion-blog/articles/speed-to-value-delivery-optimization-with-data-virtualization)

## [Speed to value delivery: Optimizing with data virtualization and data fabric](https://22034068.hs-sites.com/fusion-blog/articles/speed-to-value-delivery-optimization-with-data-virtualization)

Jun 6, 2022

 This article originally appeared in CDO magazine. Data and analytics have long held promise in helping organizations deliver greater value across the entire stakeholder landscape, including customers, associates, and partners. However, since the beginning of the data warehousing and BI movement, achieving business value rapidly — in alignment with windows of opportunity — has proven elusive. For an organization to be competitive in the era of digital transformation, data must be front and center — and accessible in near real-time. But many organizations are struggling with data that is deeply buried, complex to access, difficult to integrate, and inaccessible to business users. Problems like these diminish the value of your data and its ability to inform decision-making at all levels. For most organizations, it’s hard to produce value from data quickly The main challenge has been the distributive and siloed nature of the data subjects that need to be integrated to achieve business-relevant insights. Data subjects — customers, products, orders, warehouses, etc. — typically reside in different systems/databases, requiring extraction, transformation, and loading into a common database where analytics can be mounted. Often, data delivery solutions like data warehouses, self-service BI, and data lakes are used to try and unlock these data silos; however, each of these solutions presents drawbacks in terms of effort, complexity, cost, and time-to-market. That is where data virtualization comes in and delivers a holistic view of information to business users across all source systems. So what exactly is data virtualization? In its simplest form, data virtualization allows an organization to attach to its data subjects where they reside in real-time. It presents disparate data subjects through a semantic layer that enables them to be integrated on the fly to support query and analytic use cases. By eliminating the need to design and build complex routines that move data from multiple source locations into a single integrated data warehouse, products like Denodo enable organizations to compress weeks to months of data preparation time out of the idea-to-execution value stream. As a result, value delivery is significantly accelerated. Learn more about how Fusion & Denodo can help you streamline data access to support your most critical business needs >> Optimization with data fabric. While data virtualization integrates data from different sources into one layer to provide real-time access, data fabric is an approach with end-to-end architecture that allows organizations to manage massive amounts of data in different places and automates the integration process. The thing about data fabric is that it has a huge job to do and must have a robust integration backbone to do it. A data fabric must support many data sources, be compatible with several data pipeline workflows, support automated data orchestration, empower various kinds of data consumers, and more. To do this successfully, a data fabric requires powerful technologies and a solid data integration layer to access all data assets. Many in the data community believe that you must choose data virtualization OR data fabric, but that is not the case — and that solid data integration layer is an example of why. The reality is that data fabric can be operationalized through data virtualization and optimizes your modern data architecture, allowing you to move with the speed of your business. By building a model that utilizes both concepts, businesses make finding, interpreting, and using their data near seamless. Technology by itself isn’t the answer. Even with the proven results of this class of technologies, many organizations continue to struggle with traditional data management and analytic architectures and solutions. This inability to adopt new approaches for data management and analytics only serves to deprive decision makers of rapid access to insights that are necessary to support agility in the pandemic-induced, rapidly transforming digital/global economy. The solution is not just found in technology. Instead, it is found in the minds of the humans responsible for delivering data management and analytic capabilities. It is a human change management problem we face. Remember the adage, people/process/data/technology? The next frontier to be conquered is optimizing the thinking and innovation risk tolerance of stewards of data management and analytics solutions within organizations. What do you think? Is your organization facing any of these issues or trying to tackle how to deliver significant value — better, faster, cheaper, smarter? I’m happy to chat about where you’re at and how to get where you would like to be. If you want to talk, send me a note.

[![](https://22034068.hs-sites.com/hubfs/fusion/data-communities.jpg) ](https://22034068.hs-sites.com/fusion-blog/articles/data-executives-empowered-the-importance-of-the-data-community-and-elevating-the-cdo)

## [Data executives empowered: The importance of the data community and elevating the CDO](https://22034068.hs-sites.com/fusion-blog/articles/data-executives-empowered-the-importance-of-the-data-community-and-elevating-the-cdo)

Jul 22, 2021

 Almost 20 years ago, Capital One recognized the need for one person to oversee their data security, quality, and privacy, and the role of the Chief Data Officer was born. Now reports show that 68% of organizations have a CDO (Harvard Business Review, 2020). And while the role has become more common and has significantly evolved, many data executives are still struggling to get a seat at the table or bring data to the forefront of their organization. In fact, in a recent survey, only 28% of respondents agreed that the role was successful and established. Company leaders agree that there needs to be a single point of accountability to manage the various dimensions of data inside and outside of the enterprise, including the quality and availability of that data. But now we are at a crossroads — what is the best way to align the work that the CDO does with the strategy of the business as a whole? The reality is that CDOs often struggle to find the internal and external support and resources needed to educate others to align with the organization’s goals. Implementing enterprise data governance, data architecture, data asset development, data science, and advanced analytics capabilities — such as machine learning and video analytics — at scale, is not an easy task. To be successful, data executives need support, resources, and communities focused on the elevation of data. We are proud to continue to help these communities come to life for the benefit of our colleagues and clients, establishing local resources here in the Midwest with global scale, reach, and impact. Read on as Mark Johnson, our Executive Leader for Data Management and Analytics, provides insight on the current state of data and the CDO, and provides details on multiple opportunities for data leaders of different levels to get more involved in the data community. Q: How has the role of data changed/evolved for organizations? The reality is that information is everything. This global pandemic proved that to many organizations. For some, it showed that their digital network was ready, and they were aptly prepared to take on COVID. For others, it has forced them to recognize their own immaturity with data and analytics. On its own, managing data is not exciting — the information just sort of exists. To give data value, you have to put it to use. And so, I think we are going to see the Chief Data Officer and/or Chief Data Analytics Officer really find their own in the coming years. It’s time for their seat at the table. The C-suite is now asking questions that can only be answered with data, and now they truly understand both the value and consequences of the data game. Q: What do you think are the biggest challenges facing CDOs/data leaders today? I think that the biggest challenge for data executives today is the acquisition of talent that is seasoned and experienced where you need them to be for your organization. Higher education hasn’t necessarily kept up with the data world, and often times it takes additional training to reach the right levels. The reality is that right now the talent is manufactured in the real world. Data executives have to be connected and equipped to mentor, train, and keep the right people. Q: You’ve mentioned that data leaders need to connect with each other. What value can people expect from these data communities? I think there is tremendous value. As we are seeing the power of data evolve in organizations, and the role of data leaders evolve as well, I think coming together to collaborate and share elevates the leader, the organization, and the view of data as a whole. In these communities, it gives people a safe space to talk about how they are doing, what they are doing, what their biggest challenges are, and what solutions are working for them. These communities have truly become both a learning laboratory and an accelerator for data. Q: As a big proponent of connecting data leaders, you have been involved in creating different opportunities for people to get together. What groups/events would you recommend, and how can people get involved? I personally have been involved with the MIT Chief Data Officer and Information Quality Symposium (MIT CDOIQ), which is such a great opportunity to start with for connection. It has developed into additional opportunities for data leaders at all levels to get involved and create the kind of community we need to truly elevate the value of data. Organizations like the CDO Magazine, the creation of CDO roundtables across the nation, and the International Society of Chief Data Officers (isCDO) all evolved from connecting data leaders and identifying common challenges. MIT CDOIQ: The International MIT Chief Data Officer and Information Quality Symposium (MIT CDOIQ) is one of the key events for sharing and exchanging cutting-edge ideas and creating a space for discussion between data executives across industries. While resolving data issues at the Department of Defense, the symposium founder, Dr. Wang, recognized the need to bring data people together. Now in its 15th year, MIT CDOIQ is a premier event designed to advance knowledge, accelerate the adoption of the role of the Chief Data Officer, and change how data is leveraged in organizations across industries and geographies. Fusion has been a sponsor of this symposium for seven years now, and we are so excited to see how the event has grown. Designed for the CDO or top data executive in your organization, this is a space to really connect with other top industry leaders. CDO Roundtables Fusion has always been focused on building community and connecting people. And when one of our clients, a Fortune 500 retailer, mentioned wanting to talk with other data leaders from similar corporations, we realized that there was a big gap here — there was no space that existed where data leaders could informally come together, without sales pitches and vendor influence, and simply talk. That’s how the CDO roundtables were born — a place that allows data leaders to get to know each other, collaborate, accelerate knowledge growth, and problem solve. We just started two years ago in Cincinnati, but now, now we’ve expanded to multiple markets including Indianapolis, Columbus, Cleveland, Chicago, and Miami. These groups are designed for your CDO/CDAO and truly create an environment for unfiltered peer-to-peer discussion that helps solves data leadership challenges across industries. If you’re interested in joining one of these roundtables or starting one in your market, email me or message me on LinkedIn. I’m here and ready to get these roundtables started with executives in as many communities as I can. The more communities we have, the more data leaders and organizations we can serve. International Society of Chief Data Officers (isCDO) Launched out of the MIT CDOIQ symposium, the isCDO is a vendor-neutral organization designed to promote data leadership. I am excited to be a founding member of this organization, along with our Vice President of Strategy, David Levine. Our ultimate goal is to create a space that serves as a peer-advisory resource and enables enterprises to truly realize the value of data-driven decision making. With multiple membership options available, isCDO is the perfect opportunity for data leaders looking to connect with their peers and gain a competitive advantage by focusing on high-quality data and analytics. CDO Magazine I am really proud to be a founder of the CDO magazine, as it really is a resource for all business leaders, not just the CDO. We designed the magazine to be a resource for C-suite leaders — to educate and inform on the value proposition, strategies, and best practices that optimize long-term business value from investments in enterprise data management and analytics capabilities. Check out the publication here. And if you’re interested in contributing content or being interviewed, let me know at mjohnson@fusionalliance.com. Closing: The role of the CDO is integral to organizations, but it’s still evolving. Now more than ever, it is important that data leaders come together to collaborate and problem-solve. Fusion is excited to be a part of each of these initiatives, and we are committed to being an agent of change in the communities we serve and beyond. By connecting global thought leaders we believe that organizations will realize the value of data to power their digital transformation. If you’re interested in joining any of these data communities or just have questions, feel free to reach out to Mark via email or on LinkedIn.

[![](https://22034068.hs-sites.com/hubfs/fusion/big-data.jpg) ](https://22034068.hs-sites.com/fusion-blog/articles/the-rise-of-big-data-technologies-and-why-it-matters)

## [The rise of big data technologies and why it matters](https://22034068.hs-sites.com/fusion-blog/articles/the-rise-of-big-data-technologies-and-why-it-matters)

Mar 24, 2021

 Getting the right data to the right people at the right time is the name of the game in today’s demanding marketplace. Every company has to find a way to harness big data and use it to drive growth. And if your organization isn’t talking big data, you are at a competitive disadvantage. This article covers a top-level view of big data’s evolution and key components. It can help you understand the importance of big data and technologies that are essential to its discussion. With this foundation, you can proceed to the next step — addressing what to do with your data and how. Just how much data exists? Every passing moment, the pace of data creation continues to compound. In the time it takes you to read these few paragraphs there will be: more than 200 million emails sent millions of dollars in e-commerce transacted 50 hours of YouTube videos uploaded millions of Google searches launched tens of millions of photos shared Every few minutes, this cycle repeats and grows. In 2019, 90% of the world’s digital data had been created in the prior two years alone. By 2025, the global datasphere will grow to 175 zettabytes (up from 45 zettabytes in 2019). And nearly 30% of the world’s data will need real-time processing. Over the last decade, an entire ecosystem of technologies has emerged to meet the business demand for processing an unprecedented amount of consumer data. What is big data? Big data happens when there is more input than can be processed using current data management systems. The arrival of smartphones and tablets was the tipping point that led to big data. With the internet as the catalyst, data creation exploded with the ability to have music, documents, books, movies, conversations, images, text messages, announcements, and alerts readily accessible. Digital channels (websites, applications, social media) exist to entertain, inform, and add convenience to our lives. But their role goes beyond the consumer audience — accumulating invaluable data to inform business strategies. Digital technology that logs, aggregates, and integrates with open data sources enables organizations to get the most out of their data, and methodically improves bottom lines. Big data can be categorized into structured, unstructured, and semi-structured formats. The development of modern data architecture Until recently, businesses relied on basic technologies from select vendors. In the 1980s, Windows and the Mac OS debuted with integrated data management technology, and early versions of relational database engines began to become commercially viable. Then Linux came onto the scene in 1991, releasing a free operating system kernel. This paved the way for big data management. What is big data technology? Big data technologies refer to the software specifically designed to analyze, process, and extract information from complex data sets. There are different programs and systems that can do this. Distributed file systems In the early 2000s, Google proposed the Google file system, a technology for indexing and managing mounting data. A key tenet to the idea was using more low-cost machines to accomplish big tasks more efficiently and inexpensively than the hardware on a central server. Before the Information Age, data was transactional and structured. Today’s data is assorted and needs a file system that can ingest and sort massive influxes of unstructured data. Open-source and commercial software tools automate the necessary actions to enable the new varieties of data, and its attendant metadata, to be readily available for analysis. Hadoop Inspired by the promise of distributing the processing load for the increasing volumes of data, Doug Cutting and Mike Cafarella created Hadoop in 2005. The Apache Software Foundation took the value of data to the next level with the release of Hadoop in Dec. 2011. Today, this open-source software technology is packaged with services and support from new vendors to manage companies’ most valuable asset: data. The Hadoop architecture relies on distributing workloads across numerous low-cost commodity servers. Each of these “pizza boxes” (so called because they are an inch high and less than 20 inches wide and deep) has a CPU, memory, and disk storage. They are simple servers with the ability to process immense amounts of various, unstructured data when running as nodes in a Hadoop cluster. A more powerful machine called the “name node” manages the distribution of incoming data across the nodes. By default, data is written to at least three nodes and might not exist in its entirety as a single file in any one node. Below is a simple diagram that illustrates the Hadoop architecture at work. Open source software The majority of enterprises today use open source software (OSS). From operating systems to utilities to data management software, OSS has become the standard fare for corporate software development groups. Serving as a progressive OSS organization, Apache Software Foundation is a non-profit group of thousands of volunteers who contribute their time and skills to building useful software tools. As the creators, Apache continuously works to enhance Hadoop code — including its distributed file system called Hadoop Distributed File System (HDFS) — as well as the code distribution and execution features known as MapReduce. Within the past few years, Apache released nearly 50 related software systems and components for the Hadoop ecosystem. Several of these systems have counterparts in the commercial software industry. Vendors have packaged Apache’s Hadoop with user interfaces and extensions, while offering enterprise-class support for a service fee. In this segment of the OSS industry, Cloudera, Hortonworks, and Pivotal are leading firms serving big data environments. Now software systems are so tightly developed to the core Hadoop environment that no commercial vendor has attempted to assimilate the functionality. The range of OSS systems, tools, products, and extensions to Hadoop include capabilities to import, query, secure, schedule, manage, and analyze data from various sources. Storage Corporate NAS and SAN technologies, cloud storage, and on-demand programmatic requests returning JSON, XML, or other structures are often secure repositories of ancillary data. The same applies to public datasets — freely available datasets, in many cases for economic activity by industry classification, weather, demographics, location data, and thousands more topics. Data of this measure demands storage. Distributed file systems greatly reduce storage costs while providing redundancy and high availability. Each node has its local storage. These drives don’t require speed or solid-state drives, commonly called SSDs. They are inexpensive, high-capacity pedestrian drives. Upon ingestion, each file is written to three drives by default. Hadoop’s management tools and the Name Node monitor each node’s activity and health so that poorly performing nodes can be bypassed or taken out of the distributed file system index for maintenance. The term “data lake” describes the vast storage of different types of data. These vastly different data sources are in a minimum of a dozen different file formats. Some are compressed or zipped. Some have associated machine data, as found in photos taken with any phone or digital camera. The date, camera settings, and often the location are available for analysis. For example, a query to the lake for text messages that included an image taken between 9 p.m. and 2 a.m. on Friday or Saturday nights in Orlando on an iPhone would probably show fireworks at Disney World in at least 25% of the images. Administration of big data initiatives The enterprise administration of applications — their storage requirements, security granularity, compliance, and dependencies — required Hadoop distributions to mature these capabilities in the course of becoming a managed service to an enterprise (like those from Cloudera and Hortonworks). In the graphic above, you can see a view of Hadoop’s place among other software ecosystems. Note that popular analysis tools (below) are valuable in developing big data solutions: Excel and Tableau databases such as SQL Server and Oracle development platforms such as Java and Informatica Data Quality Administration through Cisco and HP tools is common. Further development of big data Commercial software companies have begun connecting to Hadoop, offering functionality such as: data integration quality assessment context management visualization and analysis from companies such as IBM, Microsoft, Informatica, SAP, Tableau, Experian, and other standard carriers. Analytics and data science Analytics is the endgame for developing a big data environment. The rise of big data has given credence to a new resource classification, the data scientist — a person who embodies an analyst, technologist, and statistician all in one. Using several approaches, a data scientist might perform exploratory queries using Spark or Impala, or might use a programming language such as R or Python. As a free language, R is rapidly growing in popularity. It is approachable by anyone who is comfortable with macro languages such as those found in Excel. R and its libraries implement statistical and graphical techniques. Moving to the cloud Cloud computing is very different from server-class hardware and software. It involves cloud storage, multi-tenant shared hosts, and managed virtual servers that are not housed on a company’s premises. In cloud environments, an organization does not own equipment, nor does it employ the network and security technologists to manage the systems. Cloud computing provides a hosted experience, where services are fully remote and accessed with a browser. The investment to build a 10- or 20-node HDFS cluster in the cloud is relatively small compared to the cost of implementing a large-scale server cluster with conventional technologies. The initial build-out of redundant centers by Amazon, Microsoft, Google, IBM, Rackspace, and others has passed. We now have systems available at prices below the cost of a single technician. Today, cloud computing fees change rapidly with pricing measured by various usage patterns. Conclusion: The rise of big data is evident Big data is not a fad or soon-to-fade trending hashtag. Since you began reading this article, more than 100 million photos have been created, with a sizeable portion having a first-degree relationship to your industry. And the pace of data creation continues to increase. The distribution of computing processes can help organizations to gain a 360-degree view of their customers through big data collection and analysis. And companies that embrace big data technologies and solutions will rise ahead of their competitors. Big data technologies are becoming an industry standard in finance, commerce, insurance, healthcare, and distribution. Welcoming big data technologies and solutions is key in the optimization and continued growth going forward. Companies that embrace data solutions can continue to improve management and operational processes and create a competitive advantage to withstand an ever-evolving marketplace.

<https://22034068.hs-sites.com/fusion-blog/articles/keep-the-wealth-in-your-financial-institution>

## [Customer retention strategies for banks: 5 tips for keeping the wealth in your institution](https://22034068.hs-sites.com/fusion-blog/articles/keep-the-wealth-in-your-financial-institution)

Aug 25, 2020

 A massive storm is brewing in the banking, financial services, and insurance industries, and when it strikes, it will be devastating to the unprepared. That storm is the unprecedented transfer of wealth, $3.9 trillion worth, that will be passed from the hands of older generations to younger in the next eight years or so. The rains have already started to trickle, but when they come in full force, if your organization hasn’t already connected with younger generations, you’ll see millions of dollars in wealth walk right out your door. If your bank doesn't have a plan in place for customer retention, it’s not too late to take action. Consider that millennials (born circa 1981-1997, also called Gen Y), are now the largest generation, accounting for over 25% of the population. They are followed by Gen Z (born circa 2000-present), those born with digital devices in their hands, who comprise more than 20% of the population. The potential purchasing power of these generations combined is something that can make or break banks, wealth management firms, and insurance companies. Yet most businesses in these industries still don’t have a game plan to connect with an entire population. Will your company be different? The problem is complex, but no matter where you stand, a solution is within your reach if you create a strategy informed by data and insights that has a clear road map to success. Here are five tips to building successful customer retention strategies for your bank, so you can emerge strong on the other side of the impending wealth transfer. 1. Understand the challenges of banking for millennials Recognize that this is a whole new audience you’re dealing with. The old ways won’t work in the new economy of connected consumerism. A 360-degree view of your current customers will help you gain insights into what the older generation wants, but with an eye towards the future consumers of your brand. They’re not like baby boomers (born 1946-1964) or Generation Xers (born 1965-1979). This newer generation sees things differently than their parents and grandparents did. Get to know this younger audience on their terms and understand why they have different belief and value systems, why they view traditional institutions skeptically. Examine the world from their eyes. They’ve seen that industry giants who their elders once perceived as invincible (e.g., Lehman Brothers) are now gone. Or that others, like Wells Fargo, AIG, and Countrywide, had to be rescued by the government from the brink of bankruptcy, with taxpayers footing the bill. They’ve seen the effects of parents being laid off after years of loyal service to a corporation. They know families who lost their homes when the housing bubble burst. Can you blame them for being leery of traditional institutions? An Androit Digital survey examining millennials’ brand loyalty reported that 77% said they use different criteria to evaluate brands than their parents do. Are you aware of what criteria they are using to evaluate your brand? If not, you need to arm yourself with answers. Research shows that younger generations frequently turn to friends, independent online research, reviews, and social media for decision making. For example, an astounding 93% of millennials read reviews before making a purchase, and 89% “believe friends’ comments more than company claims,” according to an IBM Institute for Business Value survey. Your future hinges on understanding these behaviors. A report by Gallup on the insurance sector revealed, “Millennials are more than twice as likely (27% vs. 11% respectively) as all other generations to purchase their [insurance] policies online rather than through an agent.” Online purchasing is far from the mainstream among insurance consumers overall: “74% originally purchased with an agent vs. 14% online – but if this trend among millennials continues to grow, it could substantially change the way insurance companies interact with customers in the coming years,” the report stated. Likewise, “Banks are losing touch with an entire dominant generation,” according to Joe Kessler, president of Cassandra Global. The Cassandra Report cited that 58% of young adults said they would rather borrow money from friends or family instead of a traditional institution. Two-thirds of the respondents said it is “hard to know where to learn about what financial services they might need.” In other words, when it comes to banking, millennials don’t know who to trust. Begin the process of getting to know this younger clientele by conducting research that will help you gain insights into what they stand for, how and where they interact, and what their expectations are of your industry, your company, and your brand. By evaluating that data, you will be able to set the process for communicating with and building different ways to engage with these new young consumers. Your interactions and communications must be seamless and easy and reflect that you can talk in their terms. You’ll need to look at this emerging demographic with a “digital lens” because this is how millennials engage with brands. What are those channels, what are their preferences? What other services can you make available in a seamless and frictionless and customized way? If you don’t take the time to get to know your audience, you won’t be able to lay the foundation for a successful strategy to engage them. 2. Make young customer retention your bank’s primary mission Younger generations, millennials especially, are driven by a different set of values. They want a work/life balance. They like to donate money. They don’t want a lot of stuff. They like to travel. They want to experience life. They question long-standing rules that don’t make sense to them. So, develop your business strategy around a purpose or a mission – one that they will connect with. Build upon the information you learned about your younger customers in tip #1, then map this customer’s journey with behavioral analytics. Evaluate the digital channels and content that your younger clients find compelling. Now you can create a strategy and roadmap to engage these customers. 3. Build your customer experience for different audiences A strong customer experience (CX), one that creates loyalty, is one that is personalized, timely, relevant, appropriate, and built on trust. The more customizable the user experience, the better. According to Janrain, 74% of online users are frustrated with brands that provide content that doesn’t reflect their personal interests. You know users want to be recognized on their terms, but you have a problem. How do you build a single CX that addresses vastly different generations with different behaviors and interests? Is there a way to reconcile their differences via a single CX? The answer is no. For the time being, you need to develop both. If someone tells you differently, beware. Think about it. In wealth management, banking, and insurance, the older generation still holds the money and keeps the lights on for your business. The newer generation will get that money within a decade, but if you go full-throttle and build a single, mobile-first CX, you’re going to alienate the people holding the purse strings. In the next few pivotal years, your bank’s customer retention will be heavily dependent on how well you address each audience on their own terms. How to cater to older generations Older folks prefer offline channels, like walking into a branch, agency, or brokerage firm. They like to do business face to face or via phone conversations with tellers, bankers, agents, and wealth advisors. Online, they like having a “control panel” style experience on a desktop, such as you might find with financial trading platforms. This is how you build trust and timely, relevant, personalized experiences. Online, build a web portal to appeal to the interests, needs, and communications preferences of the older generation. The younger generation will use the web portal now and then, but that is not going to be the experience they associate with your brand – because you’ll give them their own. How to cater to younger generations Give the younger generation mobile apps and SMS communications. With over 87% of millennials saying they are never without their phone, this is where you should reach them. They have no interest in stepping foot in a building that feels like an institution or talking to some random agent, broker, or salesperson when they can do everything quickly and effortlessly on a mobile device. Take the information you learned in tips #1 and #2 and build strong loyalty, providing timely, relevant, personalized, and appropriate experiences on a digital dimension. As you build a CX specifically tailored to banking for millennials, you’ll find you can gain loyalty on their terms because you’ll be able to interact in a more agile, nimble, and personalized way. The older generation will probably use the mobile app for simple tasks like checking information and balances, but they’re going to associate their comfort with your brand with the CX they use most – the desktop. Two CXs could be the right solution for today’s transitioning market, but keep in mind that there are additional channels through which you can build loyalty with these younger audiences across the digital landscape. For example, you can share educational, informative content through social media channels. 4. Knowledge transfer to the younger generation Everyone in wealth management, insurance, and financial services already has a foot in the door with the younger generation. That connection is the strong relationship between existing older customers and their offspring. Leverage it. First, understand that the older generation wants to take care of the younger ones by leaving money to them, but they are worried that the next generation doesn’t have the knowledge or discipline to hold onto and grow that money. There are so many stories of young people, like athletes or celebrities, getting rich quickly, getting bad advice about money, and then squandering it all away. What if their children make the same mistakes? Help address that fear and protect those kids by arming your older customers with educational tools on how to prevent this from happening. For this CX, you’ll need to develop portals and educational content, manage and market that content, and make it come to life in an updated website (geared to the older generation) that features whitepapers, articles, or videos, such as “Talking to Your Children About Money 101” and the like. Educate this audience on how to talk about the benefits of insurance or long-term investment strategies and provide them with incentives to set up meetings with themselves, their offspring, and you. The younger generation isn’t interested in talking to an institution, but they will listen to the advice of the parent or grandparent giving them this money. Let the parents and grandparents have meaningful conversations that hold much more weight than your business sending a bulk email to junior that says, “Invest in an IRA.” Now when members of the younger generation, the recipients of transferred wealth, decide to check out your company on the advice of their parents or grandparents, they will access your relevant app that speaks their language and addresses things of interest to them. They’ll soon figure out that you’re not some stodgy institution and will be much more open to a discussion when their parents suggest a conversation with your company’s brokers, advisors, or agents. This is how the knowledge transfer will occur organically, growing your bank’s customer retention along the way as you build a relationship of loyalty and trust. You not only will give the benefactors peace of mind that their offspring will be good stewards of their fortune when the time comes, but you’ll keep the money in-house because you took time to connect with and earn the trust of the young beneficiaries. 5. Make use of emerging technologies in banking to satisfy the ever-changing digital landscape At this point, you know you could benefit from two CXs. The web platform focuses on the needs and concerns of the older generation that holds the wealth today. The mobile platform addresses the younger person who will inherit the wealth, providing guidance, teaching the basics of how to invest or buy insurance, and will be chock full of quizzes, games, personalized spreadsheets, automated tools, and more. The challenge is that when the older generations pass on, the desktop experience will be moot. You don’t want to have to rebuild all the technology infrastructure that you worked so hard to establish. The answer? Don’t build applications or tools – build platforms for the future that can be adapted as the younger generation takes over and as mobile-first interactions become predominant five years from now. Don’t overlook the fact that more cost-effective emerging technologies in banking, such as infrastructure in the cloud, will be a necessary ingredient for success. Banks and insurance companies are reluctant to get in the cloud, but if you understand that most applications are going to be in the cloud five years from now, you understand the critical nature of developing these capabilities today. The cloud enables rapid changes to meet market and customer demands. It is flexible and nimble. You pay for what you use, can pay for service or infrastructure, and simultaneously increase security and reliability. To those unfamiliar with the cloud, security can be a scary proposition. However, with major cloud providers like Microsoft and Amazon employing an army of experts to ensure security and regulatory compliance, the cloud is safer from a security standpoint than most on-premises data storage. While 85% of companies using the cloud report they are confident their providers are able to provide a secure environment, 90% of IT managers reported they are not confident in their own companies’ ability to detect security problems internally. If you’re building a flexible technology platform with the right digital CXs, infrastructure that looks to the future and cloud capabilities, then your organization will be positioned for success when the wealth transfer hits in the next decade. Final thoughts on customer retention strategies for banks There are more than 75 million millennials out there spending $600 billion every year, and that number is only going to increase. They are graduating from college with massive amounts of debt, face a precarious job market, and are typically naïve about financial matters and insurance.The companies who aggressively work to offer practical tools and advice on banking for millennials are the ones who will outperform their competition in the future. It’s not too late, but you cannot wait to take action. If a business does not begin building the bridge between current wealth owners and soon-to-be wealth recipients until after the wealth-transfer process has begun, it will experience a devastating economic blow and get left behind by those who have embraced this shift. The ball is in your court Everyone has predicted that the landscape of the wealth management, banking, and insurance markets will change dramatically due to the digital disruption and younger generations, but with the right strategy in place, your organization can emerge as a leader. Look at this as an opportunity to differentiate. A digital strategy will be the key to your success. Don’t look at digital as an application. Digital is the way all future generations will engage and interact. Leverage it today and do it well to tie the present with the future. Your formula for success is to create an actionable plan that is both informed and driven by insights and data on what people buy, how, what they expect, how they feel, and whether the experience is personalized, relevant, and timely. You need to understand your audience and use those insights to feed a strategy that ties into the mission and purpose of your customers. Bring your strategy to life in a digital channel that sits on top of flexible technology. Measure your customers’ experiences and level of engagement with your brand, and then make adjustments, continually working off of research and data. Follow this formula, and eight years from now, you’ll be the organization that is reaping the rewards because you understood how to keep millions of dollars from leaving your company. Need help improving your customer retention in banking? Let us know.