Cloud Hosting

Managing data can be challenging in any environment. But data management in the cloud is especially difficult, given the unique security, cost and performance issues at play. With that reality in mind, here are some tips to help IT teams optimize cloud data management and strike the right balance among the various competing priorities that shape data in public, private or hybrid cloud environments.

Before delving into best practices for cloud data management, lets briefly discuss why managing data in the cloud can be particularly challenging. The main reasons include:

Those are the problems. Now, lets look at five ways to tackle them.

A basic best practice for striking the right balance between cloud storage costs and performance is to use data storage tiers. Most public cloud providers offer different storage tiers (or classes, as they are called on some clouds) for at least their object storage services.

The higher-cost tiers offer instant access to data. With lower-cost tiers, you may have to wait some amount of time--which could range from minutes to hours--to access your data. Data that doesnt require frequent or quick access, then, can be stored much more cheaply using lower-cost tiers.

For many teams, object storage services like AWS S3 or Azure Blob Storage are the default solution for storing data in the cloud. These services let you upload data in any form and retrieve it quickly. You dont have to worry about structuring the data in a particular way or configuring a database.

The downside of cloud object storage is that you usually have to pay fees to interact with the data. For instance, if you want to list the contents of your storage bucket or copy a file, youll pay a fee for each request. The request fees are very small--fractions of a penny--but they can add up if you are constantly accessing or modifying object storage data.

You dont typically have to pay special request fees to perform data operations on other types of cloud storage services, like block storage or cloud databases. Thus, from a cost optimization perspective, it may be worth forgoing the convenience of object storage in order to save money.

One of the key security challenges that teams face when managing cloud data is the risk that they dont actually know where all of their sensitive data is within cloud environments. It can be easy to upload files containing personally identifiable information or other types of private data into the cloud and lose track of it (especially if your cloud environment is shared by a number of users within your organization, each doing their own thing with few governance policies to manage operations).

Cloud data loss prevention (DLP) tools address this problem by automatically scanning cloud storage for sensitive data. Public cloud vendors offer such tools, such as Google Cloud DLP and AWS Macie. There are also third-party DLP tools, like Open Raven, that can work within public cloud environments.

Cloud DLP wont guarantee that your cloud data is stored securely--DLP tools can overlook sensitive information--but it goes a long way toward helping you find data that is stored in an insecure way before the bad guys discover it.

Data egress--which means the movement of data out of a public cloud environment--is the bane of cloud data cost and performance optimization. The more egress you have, the more youll pay because cloud providers bill for every gigabyte of data that moves out of their clouds. Egress also leads to poorer performance due to the time it takes to move data out of the cloud via the Internet.

To mitigate these issues, make data egress mitigation a key priority when designing your cloud architecture. Dont treat egress costs and performance degradations as inevitable; instead, figure out how to store data as close as possible to the applications that process it or the users who consume it.

In addition to allowing you to store data, all of the major clouds now also let you process it using a variety of managed data analytics services, such as AWS OpenSearch and Azure Data Lake Analytics.

If you want to analyze your data without having to move it out of the cloud (and pay those nasty egress fees), these services may come in handy. However, youll typically have to pay for the services themselves, which can cost a lot depending on how much data you process. There may also be data privacy issues to consider when analyzing sensitive cloud data using a third-party service.

As an alternative, you can consider installing your own, self-managed data analytics platform in a public cloud, using open source tools like the ELK Stack. That way, you can avoid egress by keeping data in the cloud, without having to pay for a third-party managed service. (Youll pay for the cloud infrastructure that hosts the service, but that is likely to cost much less than a managed data analytics service.)

The bottom line here: Managed cloud data analytics may be a useful tool, but deploy them wisely.

Like many other things, data management is just harder when you have to do it in the cloud. The good news is that, by being strategic about which cloud storage services you use, how you manage data in the cloud and how you factor data management into your cloud architecture, you can avoid the cost, performance and security pitfalls of cloud data.

Read more from the original source:
5 Ways to Improve Data Management in the Cloud - ITPro Today

To ensure your data center design is modular and scalable, it is essential to select scalable equipment. Switchgear, uninterruptible power supplies (UPS), power distribution units (PDU), and remote power panels (RPP) are all examples of scalable equipment. (Source: ABB)

Last week, we continued our special report series on how physical infrastructure, the data center, and the cloud are keeping up with new modular solutions delivery, and streamlined operational support. In our final article in the series, wellexamine some solution architectures for scalable, modular data center designs.

With the modular market developing in the industry, some tremendous innovation and engineering design efforts have been put into solutions. The modular market is maturing, with even more large enterprises actively deploying the modular data center platform.

To that extent, there is already quite a bit of industry adoption as it relates to modular solutions:

With all of this in mind, there are still some hesitations related to modular adoption. These modular myths date back to the first generation of modular deployments. Lets examine some of these myths and where todays modular modernization and the race to data center ROI impact digital infrastructure.

MODULAR FACT

Modular solutions can be seen as intelligently applying capital to the data center in line with changing technology and IT requirements. Instead of a $50 million project on day one, ten $5 million modules can be built as they are needed. It enables the ability to add capacity to the data center incrementally.

MODULAR FACT

Heres another critical point: you dont have to worry about a lack of sub-contractors and trade professionals. Due to the nature of the design and standardized module architecture, you can have your equipment and facility up and running with minimal requirements for contractor support. The reason for this is that your equipment comes delivered as factory-built units. These modular units are pre-assembled, tested in a controlled factory environment, and delivered directly to the construction site. These efforts minimize the need for additional onsite construction and additional personnel.

As the modular data center market matures and new technologies are introduced, data center administrators will need a new way to manage their infrastructure. There will be an immediate need to transform complex data center operations into simplified plug & play delivery models. This means lights-out automation, rapid infrastructure assembly, and even further simplified management. The next iteration of DCIM aims to work more closely with modular ecosystems to remove the many challenges which face administrators when it comes to creating a road map and building around efficiencies. In working with the future of DCIM, expect the following:

MODULAR FACT

Another critical consideration is working with a modular partner that can support a healthy supply chain. When working with modular designs, make sure you have a partner that can think locally and deliver globally.

When working with modular designs, make sure you have a partner that can think locally and deliver globally.

Much like anything in the technology market, solutions continue to change and evolve. Many of the legacy perspectives on modular solutions revolve around an older generation of modular design. Today, modular data centers are more efficient, denser, and a lot easier to deploy. Lets examine some solution architectures for scalable, modular data center designs.

To ensure your data center design is modular and scalable, it is essential to select scalable equipment. Switchgear, uninterruptible power supplies (UPS), power distribution units (PDU), and remote power panels (RPP) are all examples of scalable equipment. Get this right and specifying future expansions will be time and cost-efficient.

With this in mind, lets look at some emerging Gen 2 Modular Design considerations.

Digitalization within the modular industry is a significant design consideration for Gen 2 modular designs. Systems of this nature are much more scalable because changes to the configuration can be done remotely using software, as opposed to changing out hardware or reassembling wiring.

IEC 61850 is a well-established communications standard for substation automation. The high reliability, integrated diagnostics, fine selectivity, shorter fault reaction times, and better fault tolerance delivered by IEC 61850 make it ideal for data center power infrastructure.

IEC 61850 AND MODULAR DATA CENTERS

The world is experiencing a data explosion. Not only is the quantity of data increasing at a dizzying rate, but the extent to which society relies on that data is also growing by the day. These trends elevate the data center to the status of critical infrastructure in many countries. If a data center fails, chaos ensues, which makes a reliable power supply indispensable. Generally, data centers have well-thought-out power backup provisions such as uninterruptible power supplies (UPSs), diesel generators, etc. By employing IEC 61850-enabled devices and IEC 61850-based GOOSE (generic object-oriented substation event) communication to automate the data center power infrastructure, significant improvements can be made: better power supply reliability, greater operational control, and reduced cost, for example.

GEN 2 MODULAR CONCEPTS AND AUTOMATION

Working with the next iteration of modular data center design means eliminating wasteful processes and operations. In many cases, this means adopting new solutions around infrastructure automation.

IEC 61850 is eminently suited to data center power infrastructure automation. Using just one protocol can form the bedrock of a complete electrical design concept that includes the full protection, control and supervision system, and cybersecurity. By using optical fiber instead of copper wire, wiring costs are lowered, space requirements are substantially reduced, and safety is increased. IEC 61850 also delivers the capability to monitor and control IEDs remotely. The convenience is that devices supplied by different manufacturers can communicate with each other without custom-designed gateways or other engineering-intensive complications.

Taking a broader perspective, the IEC 61850 standard allows digitalization of the data center power system in a way that opens it to collaboration with other digital entities in the data center, such as a building management system (BMS), power management system (PMS), data center infrastructure management (DCIM) or ABB Ability Data Center Automation.

These are all essential parts of the final goal: the single plane of glass that orchestrates the entire data center. Decathlon for Data Centers, for instance, gives power and cooling visibility, and IEC 61850s open protocols allow integration of existing equipment and systems. With IEC 61850 peer-to-peer communication capabilities in components like ABBs Relion relays and Emax circuit breakers, one can go from the DCIM system controlling or supervising software to having real-time interaction with the subsystem (such as a UPS breaker) itself.

The IEC 61850 architecture is the ideal standard for data centers, as it delivers increased reliability, finer selectivity, shorter fault reaction times, and the possibility to implement fault tolerance and integrated diagnostics, as well as a host of other advantages.

Download the full report, Cloud and the Data Center: How Digital Modernization is Impacting Physical Modular Infrastructure, courtesy of ABB for two exclusive case studies and tips for getting started on the modular journey.

Follow this link:
Scalable Modular Data Centers and the Race to ROI - Data Center Frontier

Dream Sports and its brands such as Dream11, FanCode, Dream Capital have a collective user base of over 140 million. To cater to such a humongous user base with peak concurrency reaching up to 5.5+ million, the cloud became a very obvious choice for the organisation from the beginning. With so much traffic data coming through, every decision is data-driven and cloud adoption ensures that this objective is achieved in a seamless manner.

With hyper-growth that we were (and are seeing) on a year-on-year basis, we needed a highly scaled solution that can be elastic as per our traffic patterns and data volumes. We were also looking to get reliable out of the box software/infrastructure as a service so that we could focus on our core product. Cloud Technologies fitted the bill perfectly, said Abhishek Ravi, CIO, Dream Sports.

According to Ravi, some key aspects that are considered for going with the cloud are scalability, elasticity, performance, reliability, resilience, security and cost.

Cloud has really helped us to plan, develop and scale our product without worrying about the performance, availability and cost of ownership. We could quickly test out our features, scale tests in load/stress environments and ship them to our users to give them the best experience. With managed services available on the cloud, our teams could and continue to focus on core products and thus, derive maximum efficiency, Ravi added.

The company intends to remain cloud-native in the future too. With our data volumes increasing day by day and newer solutions evolving in cloud space, we want to use the cloud at the optimum.

The companys strategy is multi-cloud as it selects the right solution for the different use cases as per the requirements.

Ravi believes that a multi-cloud strategy should be well thought through so that the best cloud technology for the right use case can be provisioned. It also helps to optimise the infrastructure and thus, the cost.

In the coming months, Dream Sports aims to upscale its tech advancements and expand tech infrastructure.

We leverage Big Data, Analytics, Artificial Intelligence and Machine Learning to focus on every aspect that makes sports better. We are heavily experimenting with push architecture to serve information to the users in real-time. We are also very advanced on containerization which has reduced our infrastructure requirements drastically, Ravi maintained.

The company is also working on several tech initiatives such as a concurrency prediction model to predict hourly concurrency on the Dream11 platform, and a fraud detection system to identify & mitigate users creating multiple/ duplicate accounts on the platform to abuse referral or promotional cash bonus schemes.

To ensure a quality user experience during peak traffic, Dream Sports also stress tests every feature that is released for a smooth experience at scale. We have a testing framework that simulates any kind of traffic load with real life-like patterns. This gives a high degree of assurance that the backend would behave exactly as expected, Ravi added.

Read more here:
Why Dream11 wants to float in the cloud - ETCIO.com