DataOps began popping onto my radar last fall. First there was a startup of former Kepware people developing DataOps for manufacturing enterprises, and then it had a featured role at an IT conference.
I have mentioned the two previously, which attracted the attention of Kevin E. Kline, who is working with Sentry One. He has a heck of a bio—Principal Program Manager, Bestselling author of SQL in a Nutshell, Founder & President Emeritus, PASS.Org, and a Microsoft MVP since 2003. He pointed me to a blog he had written that explains much about the topic.
These passages are lifted from that blog to give you a taste. Check out the entire post for more details. Here is a description.
DataOps is a collaborative practice that improves integration, reliability, and delivery of data across the enterprise. It builds on the foundation of strong DevOps processes. Like DevOps, DataOps fosters communication between business functions like data platform, IT operations, business analytics, engineering, and data science. It focuses on streamlining and automating the data pipeline throughout the data lifecycle:
- Data integration—simplifying the process of connecting to disparate data sources
- Data validation—testing data to ensure that business decisions are supported by accurate information
- Metadata management—maintaining a clear understanding of the topography of the data estate, origin, dependencies, and how the data has changes over time
- Observability—capturing granular insights about data systems along with rich context to help DataOps teams better understand system behavior and performance
DataOps paves the way for effective data operations and a reliable data pipeline, delivering information that people trust with shorter development and delivery cycles.
This part discusses benefits. Later he discusses obstacles.
4 Benefits of DataOps Maturity
Terms that refer to effective collaboration are alignment, tearing down silos, “synergy,” and a newer term—interlock. These terms are prevalent in business because getting them right creates a force multiplier across departments. Imagine being in a rowboat with 10 other people, and none of them are rowing in the same direction. You might never get to where you’re trying to go.
A mature DataOps practice promotes up-front planning and construction, then automated ongoing execution. In other words, teams work together to define what will happen, and various software tools ensure that it happens the same way every time.
Similar to the benefit of collaboration, the automation of data and analytics operations removes a potential element of human unpredictability. We, as human beings, are capable of great things like free thought and reason. These abilities serve us well in many situations. However, they can introduce problems when dealing with repetitive processes that must always follow the same steps.
With a mature, documented, and automated DataOps process, plans to introduce change require fewer hands, less time, and a lower probability of introducing errors. Using this approach also makes it easier to adapt testing procedures. This effectively reduces the time it takes to move from development to production for changes.
DevOps and DataOps have emerged from Agile project management practices. Because of those roots, agility becomes table stakes in DataOps processes. Data teams that already practice Agile methodologies will find it easier to define, implement, and mature their DataOps practice.
Tim Bandos, VP of Cybersecurity at Digital Guardian set aside some time to discuss his latest work, The DG Data Trends Report. Research for the report was performed during (and as a result of) the Covid-19 pandemic to study how much sensitive corporate data was “egressing” from the security of home base.
We talked last month, but I was in the midst of five or six virtual conferences and I’m only now beginning to catch up with the accumulated pile of other interviews and reports that come my way.
Digital Guardian has developed and implemented a technology that you can procure that includes an “agent” that gives visibility into data movements within and into and out of your corporate environment. It sounds pretty cool, actually.
To set the stage for the current crisis, Bandos points to the results of the 2007-2009 financial crisis:
[The crisis] led to 37 million unemployment claims. It also resulted in a slew of trade secret theft charges. In 2013, the Department of Justice said it charged more than 1,000 defendants with intellectual property theft between 2008 and 2012.
The DG report derives from real data from organizations spanning the globe and across multiple industry verticals. It is definitely not just a survey.
Following are a few tidbits from the survey.
Since the onset of Covid-19, DG saw a 123% increase in the volume of data moving to USB drives and 74% of that data was classified according to the DLP practices. Now, much of this was taking work home. But much also this data can now not be controlled.
With employees working from their homes, data egress via all means (email, cloud, USB, etc.) was 80% higher in the first month following the World Health Organization’s declaration. More than 50% of the observed data egress was classified data.
Digital Guardian’s managed Detection & Response customers noticed a 62% increase in malicious activity, a number that in turn has led to an increase in incident response investigations—64% more than before the declaration.
Five tips to protect data
1. Issue Data Governance Policy Reminders
2. Label Sensitive Information
3. Limit Access to Sensitive Data
4. Host a Remote Security Awareness Training Session
5. Consider Deploying Virtual Desktop Infrastructure or Desktop-as-a-Service.
The news in brief: New HPE GreenLake cloud services deliver an agile, lower cost, and consistent cloud experience everywhere.
We’re living in an as-a-service and edge-to-cloud world (to paraphrase the Material Girl). When Antonio Neri assumed leadership of the storied (at least part of the storied) Hewlett Packard, he grasped both that reality and that HPE had most of the tools to get there. A couple of acquisitions, a bolstered executive leadership team, and now the unveiling. There remain more work on the financial end for him, but I think HPE is positioned for growth in this arena.
Last year at Discover, HPE pushed the GreenLake idea on us. This year, it’s capabilities and possibilities are greatly expanded. And for my industrial / production readers–this applies as much to you as to Enterprise IT. It’s getting blurry at the Edge, apps like MES are moving to the cloud (actually, probably all have moved there), and the roles of Enterprise IT and Manufacturing IT are also blurring at the edges.
It’s a new world–and I don’t mean just post-Covid.
Following is from the release:
Hewlett Packard Enterprise today announced significant advancements to the company’s edge-to-cloud platform-as-a-service strategy, through next-generation cloud services and an accelerated delivery experience for HPE GreenLake. The new HPE GreenLake cloud services, which span container management, machine learning operations, VMs, storage, compute, data protection, and networking, help customers transform and modernize their applications and data – the majority of which live on premises, in colocation facilities, and increasingly at the edge.
“Now more than ever, given current market conditions, organizations have an urgent need to connect and leverage all of their applications and data in order to transform their businesses, support their employees, and serve their customers,” said Antonio Neri, President and CEO, Hewlett Packard Enterprise. “As we enter the next phase of the cloud market, customers require an approach that enables them to innovate and modernize all of their applications and workloads, including those at the edge and on premises. By delivering a consistent cloud experience everywhere through HPE GreenLake cloud services, and software designed to accelerate transformation, HPE is uniquely positioned to help customers harness the full power of their information, wherever it resides.”
Today, organizations are at a crossroads in their digital transformation efforts. According to IDC, despite the growth and adoption of public clouds, 70 percent of applications remain outside of the public cloud. Due to several factors, including application entanglement, data gravity, security and compliance, and unpredictable costs, organizations have struggled to move the majority of the applications that run their businesses to public clouds. Forced to support two operating models, organizations face additional costs, complexity and inefficiency, limited agility and innovation, and the inability to capitalize on information everywhere.
HPE delivers a unique approach to solving this dilemma by providing HPE GreenLake cloud services to customers in the environment of their choice – from edge to cloud – with a consistent operating model and with visibility and governance across all enterprise applications and data.
HPE GreenLake cloud services also provides customers with a superior economic model. Unlike public cloud vendors, which charge customers to get data back on premises, HPE charges no data egress fees. HPE GreenLake’s flexible as-a-service model and robust cost and compliance analytics tools allow customers to preserve cash flow, control spend, and prioritize investments that are aligned to business priorities.
“HPE GreenLake gives us 100% uptime, and the predictable pricing model is already helping us cut costs,” said Ed Hildreth, Manager of IT Distributed Systems, Mohawk Valley Health System. “Thanks to the cloud-like experience, when we needed to quickly activate additional features and resources in response to the COVID-19 pandemic, we were able to easily roll this out with no time delay. We are extremely pleased with HPE GreenLake and plan to leverage this model once again for new hospitals within our health system.”
Introducing New HPE GreenLake Cloud Services for Distributed Environments
HPE now offers cloud services for containers, machine learning operations, virtual machines, storage, compute, data protection and networking. All cloud services are accessible via a self-service point-and-click catalogue on HPE GreenLake Central, a platform where customers can learn about, price, and request a trial on each cloud service; spin up instances and clusters in a few clicks; and manage their multi-cloud estate from one place. They can all be deployed and run in the customers’ environment.
Based on pre-integrated building blocks, the new HPE GreenLake cloud services are now available in small, medium, and large configurations, delivered to customers from order to run in as few as 14 days. Partners and customers benefit from pre-configured reference architectures and pricing to speed time to consuming cloud services.
HPE GreenLake is one of the fastest-growing businesses in HPE with over 4 billion USD in total contract value
- Cloud services for Containers – These new HPE GreenLake cloud services, powered by HPE Ezmeral Container Platform, provide the flexibility to run containerized applications in data centers, colocation facilities, multiple public clouds, and at the edge.
- Cloud services for Machine Learning Operations – Through HPE GreenLake, customers can subscribe to a workload-specific solution built on the HPE Ezmeral Container Platform and HPE Ezmeral ML Ops for the entire ML lifecycle.
- Cloud services for Virtual Machines, Storage, and Compute – For customers who want a private cloud experience, HPE is launching HPE GreenLake cloud services for virtual machines, storage and compute. With provisioning of instances in five clicks, these easy-to-deploy services also provide visibility into usage and spend, and active capacity planning with powerful consumption analytics in the HPE GreenLake Central management platform.
- Cloud services for Data Protection – For customers looking to modernize data protection, HPE is making data backup and recovery effortless and automated for every SLA – from rapid recovery to long-term retention. These new cloud services through HPE GreenLake include secure and efficient on-premises backup and an enterprise cloud backup service, HPE Cloud Volumes Backup, which enables backup and recovery to/from the cloud without egress costs or lock-in, and with the agility to activate data for recovery, test/dev, and analytics.
- Cloud services for the Intelligent Edge – Today, more than ever, customers are looking to reduce CapEx to simplify their budget process and better predict and manage network operational costs. Aruba’s new Managed Connectivity Services, now available as cloud services through HPE GreenLake, provide the industry’s first complete Network as a Service offering, and bring cloud agility to the edge with the recently introduced Aruba ESP (Edge Services Platform).