Unitek NetApp Training Blog

A data services makeover, that is. The worldwide beauty leader Revlon has tapped NetApp to improve its business efficiency, agility, and capabilities. The decision to use NetApp storage foundation for their IT needs will help fuel growth for the company, according to Revlon CIO and senior vice president David Gaiambruno.

“When we started our IT transformation in 2006, our overarching philosophy was to simplify. Simplification enables the speed to adapt, and speed is a competitive advantage. IT’s job is to make systems work for people, rather than people working for systems,” Giambruno said. “Leveraging NetApp as our single storage and data management foundation provides a newfound level of agility that impacts every aspect of our business. With 97 percent of our total compute running on our internal cloud built by NetApp, we wield the technology needed to turn data into information. Ultimately, that enables us to support the business and to deliver high-quality, innovative products to our consumers around the world.”

When your company delivers millions of products to more than 100 countries across six-continents, data efficiency is a big challenge. A big challenge that, when met, means bottom line results. NetApp has helped harness Revlon’s 3.6 petabytes (3.6 million gigabytes), and allowed the company turn a previously burdensome amount of data into their most important informational asset.

“Revlon is an ideal example for how IT agility leads to business success. The company understands that the right IT infrastructure can be a significant enabler of the business and create a competitive advantage,” said Julie Parrish, chief marketing officer of NetApp. “Building their infrastructure on NetApp has enabled Revlon to streamline operations and change the way employees think about and utilize data.”

Source: NetApp

No Comments »

The leading provider of electronic health records (EHRs) has given NetApp its seal of approval. MEDITECH has announced that NetApp FAS Storage is officially certified to manage its EHRs, a major vote of confidence in the storage provider’s ability to house this exceptionally critical and sensitive data. For the numerous healthcare institutions that use both MEDITECH and NetApp, this news means further consolidation and streamlining of data storage.

“Children’s Hospital of Central California has chosen NetApp as a strategic partner because of their ability to help us achieve our business objectives. Up to now, we have been able to run our entire enterprise, with the exception of MEDITECH, on NetApp,” explained Kirk Larson, chief information officer at Children’s Hospital of Central California. “The fact that we will now be able to bring our MEDITECH system onto NetApp, and eliminate the last storage silo in our business means we can ultimately better serve our patients.”

As the number of healthcare institutions that are using EHRs increases, so to does the concern over the security and accessibility of these essential data. Partnering with BridgeHead, another MEDITECH data partner, NetApp is able to deliver unmatched data recovery and backup capabilities.

“Healthcare providers today are looking for increased agility in terms of how they store, process and manage the explosion of electronic patient data. They need the ability to access this data anytime and anywhere, and it must be well protected,” said Dave Nesvisky, senior director of NetApp Healthcare. “With the new MEDITECH certification on NetApp, we can now offer our customers new solutions to enhance mission-critical EHRs.”

The bottom line is that the healthcare industry is moving to EHRs and it’s not going back. NetApp has recognized this trend and continues to to be a market leader for healthcare data management.

Source: NetApp

No Comments »

Global IT analysis giant International Data Corporation (IDC) ranked NetApp Data ONTAP as the number one branded storage Operating System (OS) for 2012. IDC compared branded storage OS options worldwide and measured the value based on market revenue and new capacity shipments. NetApp led the way in the both categories.

“This recognition from IDC further reinforces Data ONTAP’s position as the industry’s #1 branded storage OS,” said Brendon Howe, vice president of product and solutions marketing at NetApp. “With the new capabilities of clustered Data ONTAP, the biggest innovation in NetApp’s history, we’re excited about extending our leadership position. Now more than ever agility is critical to business success and clustered Data ONTAP enables this for customers by delivering increased infrastructure efficiencies, the ability to grow without limits, and continuous data access. We are seeing accelerated use of these new capabilities and today’s announcement builds on this momentum.”

Data ONTAP has been refined over 20 years to provide users with superior versatility and performance at a market-leading value. The newest version, Clustered Data ONTAP, is helping businesses simplify and seamlessly integrate their data needs. A case in point is the Atlanta-based cloud services provider Cirrity, which delivers essential services to healthcare and finance institutions, and needs scalability, functionality and reliability from an OS.

“Business continuity is hugely critical to Cirrity’s reputation,” said Dan Timko, president and chief technology officer of Cirrity. “Clustered Data ONTAP allows us to non-disruptively move our data from stack to stack as our application requirements change. This means that our customers never experience a break in service, which is important since so many of them operate in 24/7 environments. Also, as we continue to grow, we can easily scale to add just the amount of storage capacity that we need. This helps reduce our overall costs and allows us to focus our spending on other parts of the business.”

Source: NetApp

No Comments »

Okay, so I figured this out a few weeks ago and posted a few quick answers to the “How Do I align disks with ESXi server” discussions out there and told a few of my students how to do it. But I have finally had a few minutes to put together all the steps and make them available.

The problem, in case you didn’t know, was how to install the mbrtools from NetApp on an ESXi server. The mbrtools are used to align vmdks. This was fine when we had ESX servers because they had the Service Console that we could connect to and install/run the tools from. ESXi is so stripped down that it makes this task a bit more difficult. The key to the solution is that NetApp has a new version of the mbrtools from Virtual Storage Console 2.1.1 (VSC) that are made just for ESXi servers. The installation guide from NetApp says to just download the tools, but it does take few more steps than that. The following post describes how to install these tools on an ESXi server.

Before I get into all that, I will also point out that I discovered another way to do this. VMware vCenter Converter Standalone Edition 5.0 also has the capability of aligning disks during both the P2V and V2V processes. I had to make sure, though, that while going through the wizard to convert the unaligned vm I explicitly chose “Select Volumes to Copy” under “Data Copy Type” when I got to the Options Section. So now it seems you have a couple of options.

Steps to install mbrtools on ESXi server

1. Download tools from VSC 2.1.1
   
2. Upload the file to ESXi server datastore
   
3. Enable ESXi server for Shell/SSH access. This means you will have to directly connect to the ESXi server console. For example, on my Dell server I have the DRAC that I can get console access through.
4. So from the Console I hit F2 in order to “Customize System/View Logs”.
   
5. Login as the root user and your screen should now look like this
   
6. Use your keyboard to select “Troubleshooting Options”. Then using your keyboard make sure you enable “ESXi Shell” and “SSH”.
   
7. SSH to ESXi server and log on as root.
   1. Cd to /vmfs/volumes
   2. Ls to see datastores
   3. Cd to the datastore you uploaded file to
   4. Use tar to extract the files
      1. “tar xvzf mbrtools_esxi.tgz”
         
8. The files are extracted to /opt/ontap
   1. Copy the files to /opt/ontap (use the GUID value for the volume name not the friendly name.)
      1. “cp -r /vmfs/volumes/4eaf88ec-4a55fdbf-05af-0015179870f8/opt/ontap /opt/ontap”
         
9. Cd to /opt/ontap/ and ls
10. You will see the tools and you can use them just as you might have with ESX
    

Tags: How Do I align disks with ESXi serve, VMware

No Comments »

This question comes up frequently and the answer is somewhat complex.  It depends on the number of drives in the storage system and the characteristics of those drives.

Remember, if you are clustering your storage system, each head in the cluster will need its own spares.  Additionally, if you are using syncmirror, you will need spares for each storage pool.

Additionally, if your storage system has both fibre channel and SATA drives you need different spares for each technology.  Fibre channel drives cannot act as spares for SATA drives and SATA drives cannot act as spares for fibre channel drives.

Ideally, if you have disks of different speeds or different sizes, you should have spares for each size and speed.  It is possible to mix drive speeds.  For example, if you have a mixture of 10k and 15k fibre channel drives in a storage system you should allocate the different speed drives to different aggregates.  If a 10k drive fails, then a 15k spare could act as a spare.  You will not be able to use the additional performance of the 15k drive, but the raid group of 10k drives will perform as before.  You are just not getting the performance you paid for when you bought the 15k drive.  The reverse if not true.  If a 15k drive fails and the only spare available is a 10k drive, then the effective speed of the raid group of 15k drives will be reduced by the 10k drive.  This is better than no spare at all, but now you have reduced performance.

It is also possible to use a large drive to act as spare for a smaller drive.  Again, this is a supported but not recommended.  The capacity of the large drive will be reduced to the same size as the drive it is replacing.  Of course, a small drive cannot act as a spare for a larger drive

After this is becomes just a question of numbers.

You should have two spares drives in each category, up to 100 drives.  Then, NetApp recommends that you add one additional spare for each additional 84 drives.  Here is a table from TR-3437:

2 Comments »

Let’s take a look at the snap autodelete command.  We will begin by checking the autodelete status:

The command: “snap autodelete vol1 show” returns the current status of the autodelete options for vol1.

By default, autodelete is turned off, so let’s start by turning it on:

It is now turned on.  The commitment option is set to try.  Either try or disrupt are available possibilities here.  This option controls which snapshots Data ONTAP will delete if it needs to recover space in the volume.  When set to try only snapshots which are not owned by data protection utilities may be deleted.  Data protection utilities include the dump command, ndmpcopy and the mirroring utilities.  Snapshots owned by backing utilities such as LUN or volume clones are also protected.   If the option is set to disrupt then snapshots which are not locked may be deleted.

The trigger option determines when the Data ONTAP will start deleting snapshots for that volume.  Trigger can be set to volume, snap_reserve or space_reserve.  If the trigger is set to volume, then once the volume has reached 98% of its capacity, it will begin deleting snapshots.  If the trigger is set to snap_reserve, then Data ONTAP will begin deleting snapshots when the volumes snapshot reserve is using 98% of its capacity.  The space_reserve option is a little more complex.  If the trigger is set to space_reserve than Data ONTAP begins deleting snapshots once the space reserved is at 98% capacity and the volume has used all of its snap reserve capacity.

Once Data ONTAP begins deleting snapshots it will continue until it reaches the value set in the target_free_space option.  It will apply this value against the “container” that reached the 98% threshold.  By default, this is set to 20% free space.  In our example, it would continue deleting snapshots until the volume had 20% free space available.

The delete_order option controls whether Data ONTAP will delete snapshots from oldest to newest or newest to oldest.  Valid values for this option are oldest_first or newest_first.

There may be some snapshots which we would like to avoid deleting if possible.  The defer_delete option will save these snapshots if possible.   Valid options are scheduled, user_created, prefix or none.  If we set this value to scheduled then snapshots that were created by Data ONTAP’s snapshot scheduler will be the last ones deleted.  If set to user_created than manual snapshots will be the last ones deleted.  If set to prefix, then snapshots whose names begin the prefix designated by the prefix option will be saved for last.  None means will are not trying to protect any particular classification of snapshots.

Finally, if we are trying to protect a class of snapshots by their name prefix, the prefix value is a string which contains the prefix name.  This string can be up to 15 characters long.

No Comments »

With the combination of Data ONTAP 7.2 and flexible volumes we have some new options for managing space in volumes that contain LUNs.  Obviously, we still have to provide a free block when the host wants to write, but now we have a new block pool in the aggregate we can pull from.  We also have the option of automatically deleting snapshots as a source of free blocks.

If we set the fractional reserve to 0% and then set the autosize option on a volume, the volume can grow when it needs more space by pulling it from the aggregate.  Effectively the aggregate is supplying the pool of available blocks.

Effectively the aggregate is providing a pool that can be shared across all the volumes it contains.  It is also very unlikely that all the LUNs in all the volumes will suddenly become extremely active at the same time.  Therefore it may make sense to set aside a free space pool that is less than what we would have provided to support a 100% space reserve for every LUN in the aggregate.

This is especially true when we consider that the autosize option is typically combined with automatic deletion of snapshots.

The following command configures vol2’s autosize options:

In this case I have configured vol2 to grow to a max size of 1 Gigabyte in 100 megabyte steps.  I can limit the maximum size the volume can achieve and I can configure the size of the increments it uses to grow.

Automatically growing volumes is generally combined with automatic deletion of snapshots.  You can control which policy you want to implement first with the following option:

In this example vol2 will first try growing the volume before deleting snapshots.  Once the volume has reached maximum size it will start deleting snapshots to free blocks.  Remember, we are not changing the size of the LUN, we are adding fresh blocks to the volume which can be used to as provide the LUN substitute blocks for block updates or overwrites.

Which is better depends on your storage environment.  You can control this behavior individually for each volume.  If you tend to keep a lot of snapshots, perhaps some longer than necessary just to be safe, then it might be better to delete snapshots first.  If you have some extra space in your aggregate, perhaps for performance reasons, then it might be better to auto-grow.

Next we’ll take a look at how to control which snapshots are deleted.

No Comments »

We have been looking at snapshot space management primarily from a NAS perspective.  From this perspective the primary technique used to deal with storage tied to the snapshots is the mechanism of the snapshot reserve.  And we have seen that all the snapshot reserve really does is set aside some space when the volume is created so that it will be available for overwrites of blocks that are tied to snapshots.  Essentially this is used to hide space usage from snapshots, but it does not affect our ability to execute a snapshot.

The situation with SAN and LUNs drives off of these ideas, but is slightly more complex.  We will be looking at this from the perspective of traditional volumes and some new capabilities available with flexible volumes.

Essentially, the problem is the same.  Once blocks in the WAFL file system are part of a snapshot they are protected.  They become read-only blocks, so to do updates we have to be able to supply unused blocks.  There is more than one way to do this.

In the world of traditional volumes we have limited options.  Traditional volumes are also aggregates.  They own their own disks and the problem has to be resolved within this context.  This is done through space reservations and the fractional reserve.

Here is an example of the lun setup script.  One of the prompts asks do we want the LUN to be space reserved and gives the explanation that this will guarantee that writes to the LUN will never fail.

Why do we need such a guarantee?  If we are not using snapshots then we really don’t need this guarantee.  But if we are taking snapshots then some blocks within the LUN will become read-only when the snapshot is taken.  Since the host operating system is not aware of the snapshot, Data ONTAP must have a pool of blocks to supply the host for overwriting blocks held in snapshots.  This pool is set aside when the LUN is created when we say “yes” to the space reserved prompt.

By default the size of this pool is equivalent to the size of the LUN.  This means the volume must be large enough to contain both the LUN, the space reserved for overwrites and the blocks tied up by snapshots.  To support a 100 MB LUN we would need volume that was at least 200MB plus some amount to support the blocks tied up in snapshots.

We can adjust the space reservations with a parameter called the fractional reserve.  The fractional reserve is applied to space reservations.  It is set to 100% by default so the block pool is able to support the LUN even if the host were to update every block in the LUN while simultaneously every block in the LUN were locked up in one or many snapshots.  This is why we can say that data writes will never fail.  If the number of blocks locked by a snapshot were to set up the situation where the number of free blocks left in the volume were less than the amount of space reserved then the snapshot would fail.  The number of free blocks in the volume would never be less than the number of blocks allocated to the LUN.

We can reduce the amount of space reserved by adjusting fractional reserve.  The following command would set the fractional reserve to 70% instead of 100% for vol2:

This decreases the amount of space reserved, but it is possible to create a situation where there may not be free blocks available in the volume when the host tries to update a block in the LUN.  If this happens the write will fail.

With traditional volumes, space reservations combined with a fractional reserve of 100% was the only way we could be sure LUN writes would never fail.  With flexible volumes and Data ONTAP 7.2, we have some new options.

No Comments »

Eventually, as more files are changed, more and more blocks will be assigned from the snapshot reserve.  The df –k command will show the amount of space being used within the active file system and in the .snapshot directory.

There is nothing magical about the 20% number.  Sometimes we may want to decrease the percentage allocated to the snap reserve.  Sometimes we may want to increase it.  It all depends on the rate at which data changes in the volume and how long we want to keep our snapshots available.

Suppose I want to change the snapshot reserve in vol3 to 30% instead of 20%.

Notice the space allocation on vol3 has now changed.

As you can see, it is possible that more than 20% of the blocks within the volume may actually be assigned to the snapshot reserve.  This reduces the amount of space available to the active file system.  In this case we increased the space allocated to the snapshot reserve.  We could just as easily reduce the size of the snapshot reserve.

If these are NAS volumes – used for CIFS or NFS – then the creation and deletion of snapshots is probably being controlled by Data ONTAP’s snapshot scheduler.  This is configured either with the snap sched command or from filerview.  Filerview’s screen to modify the parameters for vol3 looks like this:

Notice we can change the percentage of space preallocated for snapshot blocks here as well as with the snap reserve command.

What I’m interested in here is the Number of Scheduled Snapshots to Keep options.  Logically, the longer I keep snapshots around the greater the difference will be between the blocks in the active filesystem and the oldest snapshot.  Remember that blocks pointed to by any snapshot cannot be updated – they are read-only – as long as the snapshot exists.

The snapshot scheduler controls how long the snapshots will be kept and when they will occur.  For example, weekly snapshots occur at midnight on Sunday nights.  If the scheduler is set to retain the last 6 weekly snapshots it will delete the oldest weekly snapshot after it has accumulated 6 weekly snapshots. The last 6 weekly snapshots will be maintained and older snapshots will be automatically deleted.

The same is true for nightly snapshots (which are taken at midnight every day except Sunday) and hourly backups (which are taken on the hours specified).  The number scheduled to be kept will limit how far back we can go to recover files.

Ideally equilibrium is reached, with the percentage of blocks allocated to the snapshot reserve adequate for the length of time we want to have the snapshots available.  As snapshots are automatically deleted blocks which are pointed to uniquely from that snapshot will be returned to the free space pool in the active file system.

If more blocks are needed than have been set aside by the snapshot reserve than they will be taken from the active file system.  (This is the default behavior and can be changed.) A user will see a situation where deleting files may not actually free any space.  This is because there are no available free blocks in the snapshot reserve.  Free blocks will not be available until some snapshots are deleted and, if no other snapshot points to them, the blocks will be returned for reuse.

In extreme situations, it is possible that the active file system may actually run out of blocks and it will be impossible to change files in the volume or add new files.  The storage system administrator will have to intervene, either by growing the volume or by deleting old snapshots so that blocks associated with them can be reused.

For volumes used to support NAS applications this is usually adequate.  This situation should be extremely rare.  After the system administrator deletes some snapshots or grows the volume, users can proceed normally.  For volumes that contain LUNs, the situation is more complex and   we’ll be looking at some additional options that might be useful for these volumes next time.

No Comments »

We’ve been discussing how the WAFL file system allocates space for volumes, including the   meaning of space guarantees, space reservations and fractional reserve.  Now I’d like to look a little deeper at snapshots and space management from a snapshot perspective.

Basically, when a snapshot occurs all the blocks associated with data in the volume are frozen.  Sometimes people use the term snapshot “copy”.  This is accurate in a certain sense.  The snapshot looks like a copy of the data in the file system.  For the most part, we can generally treat the snapshot as if it were a read-only copy of the data in the file system.

In fact, the blocks of data were not copied.  They were simply frozen in place.  Each snapshot creates a version of the file system.  The blocks that belong to files associated with each version cannot be altered.  Currently WAFL supports up to 255 versions, or snapshots, per volume.

After the snapshot occurs we can continue to change files and create new files in the active version of the file system.  We can do this because WAFL supports multiple virtual inode systems to track each version of the filesystem.  Blocks are allocated from free space available to the volume.  By design, WAFL does not overwrite data.  It writes updates into free blocks.  Obviously this integrates beautifully with snapshots, but it does raise an interesting question:  since we can have multiple versions of the files in a volume how do we manage space utilization of the volume?  Across all the “versions” of the file system, I may have only a single version of one file while I could have 256 versions (counting the one in the active file system) of another.

Generally the way this is done is through something called the snapshot reserve.

By default, when a volume is created, 20% of the space within the volume is assigned to the snapshot reserve.  If I want to check this for a volume – vol3 in my example – I would use the following command:

You can also check with the df command.  For example:

You can see the amount of space allocated to each volume is divided into two categories: the active file system and the .snapshot directory.  When volumes vol1, vol2 and vol3 were created 100 megabytes were allocated to each.  For each volume, this has been subdivided into approximately 80 MB for the active file system and 20 MB for the .snapshot directory.

Suppose we were to create a file in vol3.  Some blocks would be allocated to contain this file in vol3.  If we take a snapshot of vol3, there is still only one version of the file.  No blocks will be allocated from the snapshot reserve at this point.  However if we make changes to this file then there will be some blocks which are different between the file that exists in the snapshot and the file that exists in the snapshot.  The blocks associated with the file in the snapshot that are different from the blocks in the active files system will be assigned, not from the active file system, but from the snapshot reserve.

This is even clearer with the example of a deleted file.  Suppose we take a snapshot of a volume that contains some file and then we delete the files from the active file system.  The blocks containing the files cannot actually be freed because the snapshot still references them.  As long as the snapshot exists the files will still be there.  However the space occupied by the blocks will now be counted against the snapshot reserve and an equivalent amount of free space in the snapshot reserve will be allocated to the active file system.  This makes it look, from a space perspective, like the files were actually deleted when in fact they continue to be stored in the volume and are now referenced through a .snapshot directory.

No Comments »