{"id":14447,"date":"2019-03-14T17:39:51","date_gmt":"2019-03-14T17:39:51","guid":{"rendered":"https:\/\/rsoftsolution.com\/blog\/microsoft-open-sources-its-data-compression-algorithm-and-hardware-for-the-cloud.html"},"modified":"2019-03-14T17:39:51","modified_gmt":"2019-03-14T17:39:51","slug":"microsoft-open-sources-its-data-compression-algorithm-and-hardware-for-the-cloud","status":"publish","type":"post","link":"https:\/\/www.rbsitsoftwaresolution.in\/blog\/microsoft-open-sources-its-data-compression-algorithm-and-hardware-for-the-cloud.html","title":{"rendered":"Microsoft open sources its data compression algorithm and hardware for the cloud"},"content":{"rendered":"

The amount of data that the big cloud computing providers now store is staggering, so it\u2019s no surprise that most store all of this information as compressed data in some form or another \u2014 just like you used to zip your files back in the days of floppy disks, CD-ROMs and low-bandwidth connections. Typically, those systems are closely guarded secrets, but today, Microsoft <\/a> open sourced<\/a> the algorithm, hardware specification and Verilog<\/a> source code for how it compresses data in its Azure cloud. The company is contributing all of this to the Open Compute Project<\/a> (OCP).<\/p>\n

Project Zipline, as Microsoft calls this project, can achieve 2x higher compression ratios compared to the standard Zlib<\/a>-L4 64KB model. To do this, the algorithm \u2014 and its hardware implementation \u2014 were specifically tuned for the kind of large datasets Microsoft sees in its cloud. Because the system works at the systems level, there is virtually no overhead and Microsoft says that it is actually able to manage higher throughput rates and lower latency than other algorithms are currently able to achieve.<\/p>\n

Microsoft stresses that it is also contributing the Verilog source code for register transfer language<\/a> (RTL) \u2014 that is, the low-level code that makes this all work. \u201cContributing RTL at this level of detail as open source to OCP is industry leading,\u201d Kushagra Vaid, the general manager for Azure hardware infrastructure, writes. \u201cIt sets a new precedent for driving frictionless collaboration in the OCP ecosystem for new technologies and opening the doors for hardware innovation at the silicon level.\u201d<\/p>\n

\"\"<\/p>\n

Microsoft is currently using this system in its own Azure cloud, but it is now also partnering with others in the Open Compute Project. Among these partners are Intel, AMD, Ampere, Arm, Marvell, SiFive, Broadcom, Fungible, Mellanox, NGD Systems, Pure Storage, Synopsys and Cadence.<\/p>\n

\u201cOver time, we anticipate Project Zipline compression technology will make its way into several market segments and usage models such as network data processing, smart SSDs, archival systems, cloud appliances, general purpose microprocessor, IoT, and edge devices,\u201d writes Vaid.<\/p>\n

Article source: http:\/\/feedproxy.google.com\/~r\/techcrunchIt\/~3\/S_vPXFRC5pU\/<\/p>","protected":false},"excerpt":{"rendered":"

The amount of data that the big cloud computing providers now store is staggering, so it\u2019s no surprise that most store all of this information as compressed data in some <\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[19],"tags":[25],"class_list":["post-14447","post","type-post","status-publish","format-standard","hentry","category-enterprise-software","tag-enterprise"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t