级联:延迟敏感边缘智能平台

Weijia Song, Thiago Garrett, Yuting Yang, Mingzhao Liu, Edward Tremel, Lorenzo Rosa, Andrea Merlina, Roman Vitenberg, Ken Birman
{"title":"级联:延迟敏感边缘智能平台","authors":"Weijia Song, Thiago Garrett, Yuting Yang, Mingzhao Liu, Edward Tremel, Lorenzo Rosa, Andrea Merlina, Roman Vitenberg, Ken Birman","doi":"arxiv-2311.17329","DOIUrl":null,"url":null,"abstract":"Interactive intelligent computing applications are increasingly prevalent,\ncreating a need for AI/ML platforms optimized to reduce per-event latency while\nmaintaining high throughput and efficient resource management. Yet many\nintelligent applications run on AI/ML platforms that optimize for high\nthroughput even at the cost of high tail-latency. Cascade is a new AI/ML\nhosting platform intended to untangle this puzzle. Innovations include a\nlegacy-friendly storage layer that moves data with minimal copying and a \"fast\npath\" that collocates data and computation to maximize responsiveness. Our\nevaluation shows that Cascade reduces latency by orders of magnitude with no\nloss of throughput.","PeriodicalId":501333,"journal":{"name":"arXiv - CS - Operating Systems","volume":"6 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2023-11-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Cascade: A Platform for Delay-Sensitive Edge Intelligence\",\"authors\":\"Weijia Song, Thiago Garrett, Yuting Yang, Mingzhao Liu, Edward Tremel, Lorenzo Rosa, Andrea Merlina, Roman Vitenberg, Ken Birman\",\"doi\":\"arxiv-2311.17329\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Interactive intelligent computing applications are increasingly prevalent,\\ncreating a need for AI/ML platforms optimized to reduce per-event latency while\\nmaintaining high throughput and efficient resource management. Yet many\\nintelligent applications run on AI/ML platforms that optimize for high\\nthroughput even at the cost of high tail-latency. Cascade is a new AI/ML\\nhosting platform intended to untangle this puzzle. Innovations include a\\nlegacy-friendly storage layer that moves data with minimal copying and a \\\"fast\\npath\\\" that collocates data and computation to maximize responsiveness. Our\\nevaluation shows that Cascade reduces latency by orders of magnitude with no\\nloss of throughput.\",\"PeriodicalId\":501333,\"journal\":{\"name\":\"arXiv - CS - Operating Systems\",\"volume\":\"6 1\",\"pages\":\"\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2023-11-29\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"arXiv - CS - Operating Systems\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/arxiv-2311.17329\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"arXiv - CS - Operating Systems","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/arxiv-2311.17329","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

摘要

交互式智能计算应用程序越来越普遍,因此需要对AI/ML平台进行优化,以减少每个事件的延迟,同时保持高吞吐量和高效的资源管理。然而,许多智能应用程序运行在AI/ML平台上,即使以高尾延迟为代价,也会为高吞吐量进行优化。Cascade是一个新的AI/MLhosting平台,旨在解开这个谜团。创新包括传统友好的存储层,以最小的复制移动数据,以及“快速路径”,将数据和计算并置,以最大限度地提高响应能力。我们的评估表明,级联在不损失吞吐量的情况下减少了几个数量级的延迟。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
Cascade: A Platform for Delay-Sensitive Edge Intelligence
Interactive intelligent computing applications are increasingly prevalent, creating a need for AI/ML platforms optimized to reduce per-event latency while maintaining high throughput and efficient resource management. Yet many intelligent applications run on AI/ML platforms that optimize for high throughput even at the cost of high tail-latency. Cascade is a new AI/ML hosting platform intended to untangle this puzzle. Innovations include a legacy-friendly storage layer that moves data with minimal copying and a "fast path" that collocates data and computation to maximize responsiveness. Our evaluation shows that Cascade reduces latency by orders of magnitude with no loss of throughput.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
自引率
0.00%
发文量
0
期刊最新文献
Analysis of Synchronization Mechanisms in Operating Systems Skip TLB flushes for reused pages within mmap's eBPF-mm: Userspace-guided memory management in Linux with eBPF BULKHEAD: Secure, Scalable, and Efficient Kernel Compartmentalization with PKS Rethinking Programmed I/O for Fast Devices, Cheap Cores, and Coherent Interconnects
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1