cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
1247
Views
10
Helpful
2
Replies

HX native snapshots

stu2020
Level 1
Level 1

Hi

 

I have a cluster of HX220c M5 nodes and would like to know a bit more about how native snapshots are created and stored.  I have read the HX Data Platform Admin guide that explains how to create native snapshots manually or using a schedule, with the vSphere plugin but it doesn't mention how the snapshots are stored.

 

When a native snapshot is created does the HX data platform create a clone of the VM?  i.e. a 100GB VM is instantly cloned to create a 2nd 100GB VM that exists until the snapshot is deleted?  Are the 2nd and 3rd native VM snapshots complete clones (100GB each time) or differentials to the 1st?  Are snapshots compressed and de-duped?

 

Thanks

stu2020

1 Accepted Solution

Accepted Solutions

RedNectar
VIP Alumni
VIP Alumni

Hi @stu2020 ,

First of all, let me answer your last question first.

Are snapshots compressed and de-duped?

The answer is of course "YES", but a simple "Yes" does not do justice to the clever way Hyperflex (HX) does this.  And to see how HX does this, you need to understand those Native Snapshots you mentioned.

You see the Hyperflex Data Platform (HXDP) uses a distributed pointer-based file system.

Let me explain.

  • The HXDP is distributed in many ways
  1. Distributed because the image file (.vmdk) that makes up the VM is split into many pieces (i'll call them "chunks" for now) are distributed across the nodes. 
    1. within each node, the "chunks" are distributed across the collection of capacity disks in the node, with the entire "chunk" being written using a log structured file system to the disk.  And the "chunks" are already compressed before being distributed.
  2. When a VM reads or writes, the read and write requests are also distributed across the nodes, in other words, a VM's read and write requests do NOT go to the local node. In fact, the local node may not even have storage (diskless nodes)
  • The pointer based file system means that an index is kept as to where all these "chunks" are, so the system can re-create the VM image. 

Now here's the good bit when it comes to both native snapshots and clones.

When a native clone of a VM is made, the HXDP simply copies the set of pointers.

Think about that for a second. That means the clone:

  • takes no extra disk space (apart from the size of the set of pointers)
  • is inherently de-duped

That's right - de-dupe is something that is inherent in the HXDP - no extra hardware, and can't be turned off. It's just the clever way the system works.

Now finallly, back to your original questions!

When a native snapshot is created does the HX data platform create a clone of the VM?

Yes it does. And of course, it is inherently de-duped. And compressed.

What happens after the first snapshot is made is that the chunks that make up the snapshot are preserved, and as the VM progresses in time, new chunks will be added and chunks deleted - but not deleted from storage (because the original snapshot relies on them), only the pointers are changed.

Again this means that the native snapshots take minimal extra space, so the answer to your other question...

Are the 2nd and 3rd native VM snapshots complete clones (100GB each time) or differentials to the 1st?

...is that subsequent snapshots are not exactly "differentials" - that's a VMware concept. No - they are MUCH more clever than "differentials"

You see, each snapshot is really just a set of pointers - an entire VM in itself.  There are NO "differential" files.

And this is great news if you should ever need to consolidate a VM, because the VM is always in a consolidated state - to consolidate you simply remove any of the the old snapshots you don't want, a process that takes seconds, because all that is bing deleted is a set of pointers and a bunch of "chunks" that do not exist in any of the other remaining snapshots marked for deletion.

I suggest you read a little more - my answer to another question may help, and so too might a Google search for "Seven things to know to make Hyperflex Go" - The first hit is probably the one you want.

I hope this helps


Don't forget to mark answers as correct if it solves your problem. This helps others find the correct answer if they search for the same problem


RedNectar aka Chris Welsh.
Forum Tips: 1. Paste images inline - don't attach. 2. Always mark helpful and correct answers, it helps others find what they need.

View solution in original post

2 Replies 2

RedNectar
VIP Alumni
VIP Alumni

Hi @stu2020 ,

First of all, let me answer your last question first.

Are snapshots compressed and de-duped?

The answer is of course "YES", but a simple "Yes" does not do justice to the clever way Hyperflex (HX) does this.  And to see how HX does this, you need to understand those Native Snapshots you mentioned.

You see the Hyperflex Data Platform (HXDP) uses a distributed pointer-based file system.

Let me explain.

  • The HXDP is distributed in many ways
  1. Distributed because the image file (.vmdk) that makes up the VM is split into many pieces (i'll call them "chunks" for now) are distributed across the nodes. 
    1. within each node, the "chunks" are distributed across the collection of capacity disks in the node, with the entire "chunk" being written using a log structured file system to the disk.  And the "chunks" are already compressed before being distributed.
  2. When a VM reads or writes, the read and write requests are also distributed across the nodes, in other words, a VM's read and write requests do NOT go to the local node. In fact, the local node may not even have storage (diskless nodes)
  • The pointer based file system means that an index is kept as to where all these "chunks" are, so the system can re-create the VM image. 

Now here's the good bit when it comes to both native snapshots and clones.

When a native clone of a VM is made, the HXDP simply copies the set of pointers.

Think about that for a second. That means the clone:

  • takes no extra disk space (apart from the size of the set of pointers)
  • is inherently de-duped

That's right - de-dupe is something that is inherent in the HXDP - no extra hardware, and can't be turned off. It's just the clever way the system works.

Now finallly, back to your original questions!

When a native snapshot is created does the HX data platform create a clone of the VM?

Yes it does. And of course, it is inherently de-duped. And compressed.

What happens after the first snapshot is made is that the chunks that make up the snapshot are preserved, and as the VM progresses in time, new chunks will be added and chunks deleted - but not deleted from storage (because the original snapshot relies on them), only the pointers are changed.

Again this means that the native snapshots take minimal extra space, so the answer to your other question...

Are the 2nd and 3rd native VM snapshots complete clones (100GB each time) or differentials to the 1st?

...is that subsequent snapshots are not exactly "differentials" - that's a VMware concept. No - they are MUCH more clever than "differentials"

You see, each snapshot is really just a set of pointers - an entire VM in itself.  There are NO "differential" files.

And this is great news if you should ever need to consolidate a VM, because the VM is always in a consolidated state - to consolidate you simply remove any of the the old snapshots you don't want, a process that takes seconds, because all that is bing deleted is a set of pointers and a bunch of "chunks" that do not exist in any of the other remaining snapshots marked for deletion.

I suggest you read a little more - my answer to another question may help, and so too might a Google search for "Seven things to know to make Hyperflex Go" - The first hit is probably the one you want.

I hope this helps


Don't forget to mark answers as correct if it solves your problem. This helps others find the correct answer if they search for the same problem


RedNectar aka Chris Welsh.
Forum Tips: 1. Paste images inline - don't attach. 2. Always mark helpful and correct answers, it helps others find what they need.

Great answer many thanks!

Review Cisco Networking for a $25 gift card

Review Cisco Networking for a $25 gift card