Different Storage Types

In this article we will learn different storage platform.
1. File Storage
2. Block Storage
3. Object based Storage

File system based storage

File storage (aka file-based storage or file-level storage) is the type of data storage where data is stored in a hierarchical file and folder structure.

• A file is stored as a whole without breaking down the data into blocks, such as in block storage.
• Files can be stored in folders, which can then be placed in other folders in a nested structure. The directory path of the file and which folder it is stored in is needed to call up that file again from its storage location.

File Storage

Block based Storage

With block storage, files are split into evenly sized blocks of data, each with its own address but with no additional information (metadata) to provide more context for what that block of data is.

HDFS (Hadoop Distributed File System) is a block based data storage platform. A file to be stored is divided into multiple blocks and these blocks (along with replicated copies of it) are stored across multiple nodes in a Hadoop cluster. This type of storage is mainly used for map based parallel processing. Generally, text based files or even binary files like Sequence Files, Avro files, Parquet files are stored in HDFS.

Block Storage

Object based Storage

Object storage, by contrast, doesn’t split files up into raw blocks of data. Instead, entire clumps of data are stored in,

• Instead of storing data as blocks, a file is stored as an Object along with the metadata associated with the object. In another words file is not divided into chunks. Instead, entire clumps of data are stored in, as a single object. Each object consists of three things. The data (100MB file), expandable amount of metadata and a globally unique identifier.

Object Storage

• Object-based storage essentially bundles the data itself along with metadata tags and a unique identifier. The metadata is customizable, which means you can input a lot more identifying information for each piece of data. These objects are stored in a flat address space, which makes it easier to locate and retrieve your data across regions.
• Generally, files like a audio-video file, image, pdf etc are stored in an object store.
• Each file stored in S3 is associated with a bucket. A bucket is something in which a file is stored as object.
• Example : AWS S3 (Simple Storage Service) is an Object based data storage platform on cloud.

Important Points

1. You access the data using a unique URL for that data.

Thanks for Reading!

If you like my work and want to support me…

1. The BEST way to support me is by following me on Medium.
2. I share tips, tricks and insights on #softwareengineering, #dataengineering #cloud #ml on LinkedIn.
3. Feel free to give claps so I know how helpful this post was for you.

Do you want to connect with me I have started mentoring others for career and interviews at 𝐭𝐨𝐩𝐦𝐚𝐭𝐞.𝐢𝐨/𝐧𝐚𝐯𝐞𝐞𝐧𝐩𝐧