Monday, August 4, 2014

Loose Lips Sink Ships... and Navies... and Nations

I am returning from a business trip in Mainz, Germany, home of Johannes Gutenberg, who invented the modern printing press in the mid-15th century. While at an exhibit, I began considering all the benefits ushered in by Gutenberg's press. I also started thinking about the drawbacks it had compared to modern technology. It was then that I began to see many correlations to how data is still being treated by most organizations today.

The premise was simple. The printing press consisted of formatted metal blocks that could be inserted and moved into columns and rows in the press. You would then setup your printing press with all the information you wanted in that page and then you would have someone run off as many copies as you needed. Repeat for every page of data. Depending on how many pages and copies, that process would take days, weeks, or months. (It is widely held that it took Gutenberg several years to print the copies of his famous version of the Bible.) Then, after a copy was completely done, it would make its way (probably via a merchant) to someone's home to be used and then kept as part of their library (it was a sign of distinction and status to display your books, in those times).

Does this sound familiar?
  • You require someone (or several) to produce a copy of that data every time a request is made.
  • You require someone to ship that data to its final destination, or perhaps to someone else who will then get it to its final destination
  • Once your copy has made it to its destination, it indefinitely stays there...or don't really know, it's completely in the hands of the requestor.
If you are still using a bucket brigade to copy and move your data, see my last post about your Pony Express.

It's the last point that is the concern in this blog

The printing press was awesome. It solved the need of getting data out to the masses. It also was called upon to duplicate and disseminate intelligence and other sensitive communications. In those cases they relied on physical and human controls to limit the risk exposure. And most organizations are still employing this 15th century security approach to data today, and they are doing so at great peril to their self and their organization, company, or country. Sure data is encrypted from endpoint to endpoint (I hope), but we live in a time where we need to take these additional steps.

Provide the right data to the right people 

In some cases, like Analytics and Break Fix, you do need to exact copies of data. But, for non-prod use cases (where most of your data resides) you want full realstic data. You want developers to test against representative names and personal information, but not the actual names of your soldiers, or credit card numbers of your consumers. Central Command knows where the munitions are, but your contractors developing your application certainly don't need to know.

Reduce attack vectors by providing the right data to the right place at the right time, and only the right time

Allow everyone the access to the data sets they need, but keep the data centrally stored, managed, archived, and controlled. When someone has a justified need for data, provide it to them virtually for the requested duration and then take it away. You have to quit making n-copies of your data across n-number of systems in your enterprise. The more times you pass sensitive information and the longer you leave it out there, your risk of compromise grows exponentially.

Recently, I had this conversation with a US Navy Rear Admiral, "If your information is actually sensitive, your approach to data sure doesn't reflect it." He looked at me with a curious smirk, then I continued. "You're creating copies of your data all over your enterprise: Dev, Staging, Test, Break-Fix, Analytics. Most with little to no sanitization. You are relying on people to do the right thing, a la Snowden." He got it. 

While you may insist you don't have a printing press in your Data Estate (Apps, BI, DR, Tactical Deployment), your approach to data security disagrees with you. Copies of unmasked data are worse than loose lips.

For more information on how you can get rid of your printing press and to easily add these capabilities to you data security strategy, please follow these links:

Wednesday, July 16, 2014

It's Time to Ditch your Pony Express

In the "Wild West" days of the early United States, there wasn't a quick, nor reliable, way to transport data from one place to another. You would have to send and receive data, via written messages. You would then have to send that message with people heading in that general direction, in hopes that it would get there months later. If that letter contained sensitive information, they would either have to write using coded wording, or hope that the numerous "carriers" would not let curiosity get the best of them. If some nefarious soul were determined to use forceful means to acquire your message, you could bet that the letter you sent with the Smith family bound for Tascaloosa was sure to be given up quickly. To solve these problems, a few entrepreneurial fellows created the Pony Express ("PE"). The PE employed mail carriers to specialize in the expedited delivery of messages (data) and to ensure secure delivery.

The formation of the PE was revolutionary, but that concept continued to be improved upon. The PE was replaced by the telegraph and the United States Post Office, and they, along with private enterprise, gradually improved both the speed and security of the delivery of data. Fast forward to today, where most of your data delivery is done electronically. But despite hundreds of years of innovation, the Pony Express has found its way back into your application delivery projects, data center consolidations,  and disaster recovery solutions; slowing your projects, injecting defects, and skyrocketing costs.

Tell-tale signs that the Pony Express is alive and well in your Government Agency or Company:

  • When a new database copy or application environment needs to be made, you submit a request/ticket that has to pass through numerous hands and wait days/weeks/months before it is delivered.
  • When a database or application environment  data needs to be refreshed, you submit a request/ticket that has to pass through numerous hands and wait days/weeks/months before it is delivered.
  • You rely on a person/group to manually ensure production data is secured via masking/obfuscation before delivering data to non-production sources.
  • You subset data to accomplish any of the above in a reasonable amount of time or at a reasonable cost.
  • You chuckled and thought "I wish I could do that" or "I wish I could get it that fast" to any of the above.

 If none of the above apply, then I wholeheartedly thank you for being a Delphix customer. If any of the above apply, then you are a victim of the 1800's.

Whether your data is a Microsoft, Oracle, PostGres, or Sybase database; or if your data is in files on Linux, Unix, or Microsoft operating systems; waiting on multiple people to deliver your data (when they get around to it) and entrusting them to secure it is a centuries old methodology. The Delphix Agile Data Platform can deliver full read/write autonomous copies of your data within minutes in as little as 10% of the space. And not just gigabytes of data, but terabytes. (Is 5TB in 5 minutes fast enough for you?) If you choose, Delphix Agile Masking will also ensure that each one of those copies has had all of the sensitive information replaced with pseudo-data to ensure that your copies will have realistic data that will help your projects move faster at reduced risk.

The Pony Express was the first attempt along the data agility journey that has recently taken a huge leap forward despite around a decade of stunted innovation and "business as usual." It's time to put those ponies out to pasture.
For more information check out the following links: 

Wednesday, July 9, 2014

Taming the Rapids Feeding Your Data Lake

First, here is a good little page that explains Hadoop Data Lakes at a high-level.

To put that into an analogy....

Just like a real lake is fed by rivers and streams, a data lake is also fed by data rivers and data streams (Binaries, flat files, Sybase, Oracle, MSSQL, etc.) Is your data stream currently fast and large enough to handle your company or government organization's data flows?

With real rivers, when heavy rains fall or a waterway becomes choked (think "beaver dam"), the river can quickly overflow its banks and wreak considerable mayhem and damage on the surrounding ecosystem. The same thing happens with data. When you have data coming in faster than you can read, process, analyze, etc., the surrounding environment can quickly become encumbered or disrupted (i.e. storage exhaustion, BI misinformation, application development delays, production outages). The same effects will happen when constraints restrict your data flow (ticketing system handoff delays between departments, inability to quickly refresh full data sets, cumbersome data rewind processes, etc.) And for every production data river, you will have 4-10 non-production tributaries (Dev(s), Test, Int, QA, Stage, UAT, Break/Fix, Training, BI, etc.). Time to build an ark.

The ebbs and tides of data are going to come and are often influenced by external factors beyond our control. The best you can do is be prepared and agile enough to adapt to the weather and adverse conditions. By virtualizing your non-production data sources with Delphix, you have now "widened you banks" by 90%. You have enabled your engineers to develop "flood control systems" two times faster, enabling your systems to quickly adapt to fast-evolving needs. By allowing them to test with full virtual environments, and not some simulation or sample data, you allow your engineers to know exactly how your systems will behave when called upon in live applications. No more simply hoping the dam will hold. Hope is not a strategy.

And those data rivers aren't just there to look pretty and enhance your view. They are there because you are using them to harness their power to benefit your company or government organization (i.e. leveraging CRM Data), irrigate other areas (i.e. feeding Data Warehouses), and as a means of agility/mobility (i.e leveraging Business Intelligence to react to market conditions).

Delphix supports the Hadoop ecosystem by enabling you to efficiently and effectively handle the various datasources that will feed the Hadoop Data Lake, all of their necessary downstream copies (staging, identification, curation, etc), and accelerate the application projects utilizing the data sources (masked and refined, if needed). Delphix delivers the right data, in the right manner (masked/unmasked), to the right team, at the right time.

Find out more about how to lift your applications and data out of the flood plane here:

Find out how Delphix helped Bobby Durrett of US Foods quickly restore production and save countless hours and overtime in his unsolicited testimonial on his blog:

Wednesday, April 16, 2014

Margin for Innovation

Does this look familiar?

Usually when I go in to speak to my customers, I draw something like this on the whiteboard or a sheet of paper.  I explain how most of my customers are faced with plateauing budgets, while the demands and associated costs keep going up.  As new application projects and mandates come in, they become a part of the operating and maintenance budget (O&M) for the next period.  When the budget doesn't scale with the demands of the business, you eventually hit the ceiling of your funding.  From that point on, you either have to go begging for additional funding (good luck with that) or you have to start chucking things out of your portfolio ("farewell training and documentation").  At this point I usually see a room full of heads nodding in agreement and "knowing smiles" being given between Line of Business owners.  I recently had a customer stop me at this point and say "Have you been spying on us?  We had a two hour meeting on this very topic yesterday."  That was very pleasant for me to hear, but the truth is that this is pain is so severe it feels personal, but it is really universal.

The first thing to go to make room in the budget is innovation.  I use innovation as a categorical way to sum up all business improving activities that are not directed by a line of business, or absolutely necessary to support the line of business.  Though not a complete rule or definition, here is my quick and easy little telltale sign as to whether something is innovation:
"We are going to do <project/task/initiative> if...."
"If we have the money/time left over...."
...or the most telling...
"I would love to be able to do <project/task/initiative> if..."
Once you have given the pink slip to innovation, the next time you hit the budget ceiling you have nowhere else to turn to lighten the load but to projects or initiatives.  We all have to make tough choices as to which application initiatives/projects deserve our time and money, this is a part of the budgeting process.  But now that innovation is gone, you find that you are spending more money and time each day doing things the old and expensive way.  That, in turn, means that your O&M grows and competes for budget/timeline room with the new application projects and initiatives.  Some new projects will be put off until a later time when funding is available (a.k.a. indefinitely) and some in-flight struggling projects will be sent on to that great project manager in the sky ("goodbye self-service product catalog").  But just when you feel like we have coped with your loss, in comes the expectedly unexpected: Mandates.

These are new and not-budgeted application project requirements that you have no choice but to address. Affordable Care Act or "Heartbleed Bug", anyone?  Ones like I just mentioned can create such a significant development disruption, that projects can be delayed by months or years and cost additional millions of dollars. You have no choice but to comply with the mandates, which means you now have to either find a new source of funds for your application project, or you have to scrap it entirely.

Because of your over-allocated budget, bloated antiquated O&M, and the inevitable and untimely appearance of mandates (surely I am not the only one who wonders why they show up the Friday before my vacation), you have have no agility and ability to react to the demands of the business.  What you need is a "Margin for Innovation" and a "Margin for Mandates".  A Margin for Innovation allows you to get to all of those projects that you need to improve service delivery, such as developing a more intuitive user interface, building a knowledge management module for your application, or converting those manual development setup/tear down/reset tasks into automated script-driven activities.  A Margin for Mandates gives you the budgeted operating room to respond to those in-flight mandated changes that have to be completed "yesterday."

The Delphix Agile Data Platform is already helping the U.S. Government and over 100 Fortune 500 accounts deliver more application projects, and deliver those projects in half the time and at higher quality.  Delphix transforms the process and economics of your application projects and maintenance, unlocking your IT budget so that you can provide that Margin for Innovation/Margin for Mandates back to the business.

Watch a short video on how Delphix can help you deliver innovation back your business at the link below.

Follow me on Twitter @CloudSurgeon