{"id":18228,"date":"2020-04-21T22:11:34","date_gmt":"2020-04-21T22:11:34","guid":{"rendered":"https:\/\/rbsitsoftwaresolution.com\/blog\/aws-and-facebook-launch-an-open-source-model-server-for-pytorch.html"},"modified":"2020-04-21T22:11:34","modified_gmt":"2020-04-21T22:11:34","slug":"aws-and-facebook-launch-an-open-source-model-server-for-pytorch","status":"publish","type":"post","link":"https:\/\/www.rbsitsoftwaresolution.in\/blog\/aws-and-facebook-launch-an-open-source-model-server-for-pytorch.html","title":{"rendered":"AWS and Facebook launch an open-source model server for PyTorch"},"content":{"rendered":"

AWS <\/a> and Facebook <\/a> today announced<\/a> two new open-source projects around PyTorch<\/a>, the popular open-source machine learning framework. The first of these is TorchServe<\/a>, a model-serving framework for PyTorch that will make it easier for developers to put their models into production. The other is TorchElastic, a library that makes it easier for developers to build fault-tolerant training jobs on Kubernetes clusters, including AWS\u2019s EC2 spot instances and Elastic Kubernetes Service.<\/p>\n

In many ways, the two companies are taking what they have learned from running their own machine learning systems at scale and are putting this into the project. For AWS, that\u2019s mostly SageMaker<\/a>, the company\u2019s machine learning platform, but as Bratin Saha, AWS VP and GM for Machine Learning Services, told me, the work on PyTorch was mostly motivated by requests from the community. And while there are obviously other model servers like TensorFlow Serving<\/a> and the Multi Model Server available today, Saha argues that it would be hard to optimize those for PyTorch.<\/p>\n

\u201cIf we tried to take some other model server, we would not be able to quote optimize it as much, as well as create it within the nuances of how PyTorch <\/a> developers like to see this,\u201d he said. AWS has lots of experience in running its own model servers for SageMaker that can handle multiple frameworks, but the community was asking for a model server that was tailored toward how they work. That also meant adapting the server\u2019s API to what PyTorch developers expect from their framework of choice, for example.<\/p>\n

As Saha told me, the server that AWS and Facebook are now launching as open source is similar to what AWS is using internally. \u201cIt\u2019s quite close,\u201d he said. \u201cWe actually started with what we had internally for one of our model servers and then put it out to the community, worked closely with Facebook, to iterate and get feedback \u2014 and then modified it so it\u2019s quite close.\u201d<\/p>\n

Bill Jia, Facebook\u2019s VP of AI Infrastructure, also told me, he\u2019s very happy about how his team and the community has pushed PyTorch forward in recent years. \u201cIf you look at the entire industry community \u2014 a large number of researchers and enterprise users are using AWS,\u201d he said. \u201cAnd then we figured out if we can collaborate with AWS and push PyTorch together, then Facebook and AWS can get a lot of benefits, but more so, all the users can get a lot of benefits from PyTorch. That\u2019s our reason for why we wanted to collaborate with AWS.\u201d<\/p>\n

As for TorchElastic, the focus here is on allowing developers to create training systems that can work on large distributed Kubernetes clusters where you might want to use cheaper spot instances. Those are preemptible, though, so your system has to be able to handle that, while traditionally, machine learning training frameworks often expect a system where the number of instances stays the same throughout the process. That, too, is something AWS originally built for SageMaker. There, it\u2019s fully managed by AWS, though, so developers never have to think about it. For developers who want more control over their dynamic training systems or to stay very close to the metal, TorchElastic now allows them to recreate this experience on their own Kubernetes clusters.<\/p>\n

AWS has a bit of a reputation when it comes to open source and its engagement with the open-source community. In this case, though, it\u2019s nice to see AWS lead the way to bring some of its own work on building model servers, for example, to the PyTorch community. In the machine learning ecosystem, that\u2019s very much expected, and Saha stressed that AWS has long engaged with the community as one of the main contributors to MXNet and through its contributions to projects like Jupyter, TensorFlow and libraries like NumPy.<\/p>\n

Article source: http:\/\/feedproxy.google.com\/~r\/techcrunchIt\/~3\/0ziNutuqVFY\/<\/p>","protected":false},"excerpt":{"rendered":"

AWS and Facebook today announced two new open-source projects around PyTorch, the popular open-source machine learning framework. The first of these is TorchServe, a model-serving framework for PyTorch that will <\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[19],"tags":[25],"class_list":["post-18228","post","type-post","status-publish","format-standard","hentry","category-enterprise-software","tag-enterprise"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t