BenchCouncil Transactions on Benchmarks, Standards and Evaluations最新文献

Open Artificial Intelligence (AI) published an AI chatbot tool called ChatGPT at the end of November 2022. Generative Pre-trained Transformer (GPT) architecture is the foundation of ChatGPT. On the internet, ChatGPT has been rapidly growing. This chatbot enables users to discuss with the AI by inputting prompts, and it is based on OpenAI’s language model. Although ChatGPT is fantastic and produces exciting results for writing tales, poetry, songs, essays, and other things, it has certain restrictions. Users may ask the bot questions, and it will reply with pertinent, convincing subjects and replies. ChatGPT has now risen to the top of several academic agendas. Administrators create task teams and hold institution-wide meetings to react to the tools, with most of the advice being to adopt this technology. This paper briefs about the ChatGPT and its need. Further, various Progressive Work Flow Processes of the ChatGPT Tool are stated diagrammatically. Specific features and capabilities of the ChatGPT Support System are studied in this paper. Finally, we identified and discussed the significant roles of ChatGPT in the current scenario. The neural language models that form the foundation of character AI have been developed from the bottom up with talks in mind. This technology implies that the programme uses deep learning methods to analyse and produce text. The model “understands” the subtleties of human-produced natural language using vast amounts of data from the internet.

Sponsored and organized by the International Open Benchmark Council (BenchCouncil), the Bench conference encompasses a wide range of topics in benchmarking, measurement, evaluation methods, and tools. Bench’s multi-disciplinary emphasis provides an ideal environment for developers and researchers from the architecture, data management, algorithm, system, network, dataset, and application communities to discuss practical and theoretical work covering workload characterization, benchmarks and tools, evaluation, measurement and optimization, and dataset generation. The Bench conferences have been successfully held for four series from 2018 to 2021 and attracted plenty of paper submissions and participants. Bench 2022 will be held virtually on Nov. 7–9, 2022, and invites manuscripts describing original work in benchmarking, measuring, and optimizing. The conference website is https://www.benchcouncil.org/bench2022/index.html.

Financial service providers find blockchain technology useful to enhance authenticity, security, and risk management. Several institutions are adopting blockchain in trade and finance systems to build smart contracts between participants, improve efficiency and transparency, and open up newer revenue opportunities. Blockchain’s unique recording capabilities make the existing clearing and settlement process redundant. Banks and other financial entities are adopting blockchain-enabled IDs to identify people. Better results come from organisations’ capacity to foresee emerging trends in financial blockchain applications and develop blockchain functionality. The transfer of asset ownership and addressing the maintenance of a precise financial ledger. Measurement, communication, and analysis of financial information are three significant areas to be focussed on by accounting professionals. Blockchain clarifies asset ownership and the existence of obligations for accountants, and it has the potential to improve productivity. This paper identifies and studies relevant articles related to blockchain for finance. This paper focuses on Blockchain technology and its importance for financial services. Further takes up various tools, strategies, and featured services in Blockchain-based financial services. Finally, the paper identifies and evaluates the significant applications of Blockchain technology in financial services. Credit reports significantly impact the financial lives of customers. Recent data breaches demonstrate the superior security of blockchain-based credit reporting over conventional server-based reporting. Blockchain-based systems enable the faster, more cost-effective, and more customised issuance of digital securities. With its adoption, the market for investors can be expanded, costs for issuers can be reduced, and counterparty risk can be reduced due to the ability to customise digital financial instruments to the demands of investors. It uses mutualised standards, protocols, and shared procedures to give network users a single common source of truth. Participants in the business network can now more easily collaborate, manage data, and agree with this technology’s application.

Understanding the designs and performance characterizations of hot interconnects on modern data center and high-performance computing (HPC) clusters is a fruitful research topic in recent years. The rapid and continuous growth of high-bandwidth and low-latency communication requirements for various types of data center and HPC applications (such as big data, deep learning, and microservices) has been pushing the envelope of advanced interconnect designs. We believe this is high time to investigate the performance characterizations of representative hot interconnects with different benchmarks. Hence, this paper presents an extensive survey of state-of-the-art hot interconnects on data center and HPC clusters and the associated representative benchmarks to help the community to better understand modern interconnects. In addition, we characterize these interconnects by the related benchmarks under different application scenarios. We provide our perspectives on benchmarking data center interconnects based on our survey, experiments, and results.

Emerging and future applications rely heavily upon systems consisting of Internet of Things (IoT), edges, data centers, and humans-in-the-loop. Significantly different from warehouse-scale computers that serve independent concurrent user requests, this new class of computer systems directly interacts with the physical world, considering humans an essential part and performing safety-critical and mission-critical operations; their computations have intertwined dependencies between not only adjacent execution loops but also actions or decisions triggered by IoTs, edge, datacenters, or humans-in-the-loop; the systems must first satisfy the accuracy metric in predicting, interpreting, or taking action before meeting the performance goal under different cases.

This article argues we need a paradigm shift to reconstruct the IoTs, edges, data centers, and humans-in-the-loop as a computer rather than a distributed system. We coin a new term, high fusion computers (HFCs), to describe this class of systems. The fusion in the term has two implications: fusing IoTs, edges, data centers, and humans-in-the-loop as a computer, fusing the physical and digital worlds through HFC systems. HFC is a pivotal case of the open-source computer systems initiative. We laid out the challenges, plan, and call for uniting our community’s wisdom and actions to address the HFC challenges. Everything, including the source code, will be publicly available from the project homepage: https://www.computercouncil.org/HFC/.

The measurable properties of the artifacts or objects in the computer, management, or finance disciplines are extrinsic, not inherent — dependent on their problem definitions and solution instantiations. The processes of problem definition, solution instantiation, and measurement are entangled. Only after the instantiation can the solutions to the problem be measured. Definition, instantiation, and measurement have complex mutual influences. Meanwhile, the technology inertia brings instantiation bias — trapped into a subspace or even a point at a high-dimension solution space. These daunting challenges, which emerging computing aggravates, make metrology cannot work for benchmark communities. It is pressing to establish independent benchmark science and engineering.

This article presents a unifying benchmark definition, a conceptual framework, and a traceable and supervised learning-based benchmarking methodology, laying the foundation for benchmark science and engineering. I also discuss BenchCouncil’s plans for emerging and future computing. The ongoing projects include defining the challenges of intelligence, instinct, quantum computers, Metaverse, planet-scale computers, and reformulating data centers, artificial intelligence for science, and CPU benchmark suites. Also, BenchCouncil will collaborate with ComputerCouncil on open-source computer systems for planet-scale computing, AI for science systems, and Metaverse.

Scientific research communities are embracing AI-based solutions to target tractable scientific tasks and improve research work flows. However, the development and evaluation of such solutions are scattered across multiple disciplines. We formalize the problem of scientific AI benchmarking, and propose a system called SAIBench in the hope of unifying the efforts and enabling low-friction on-boarding of new disciplines. The system approaches this goal with SAIL, a domain-specific language to decouple research problems, AI models, ranking criteria, and software/hardware configuration into reusable modules. We show that this approach is flexible and can adapt to problems, AI models, and evaluation methods defined in different perspectives. The project homepage is https://www.computercouncil.org/SAIBench.

Server consolidation is one of the techniques used to increase energy efficiency in datacentres. Nevertheless, the server consolidation has an inherent trade-off between performance degradation and energy consumption which has to be quantified to be managed. In this paper, the $Ci{S}^2$ index is proposed to quantify the mentioned trade-off. We validated de use of the $Ci{S}^2$ index through real experimentation. Also, these observations lead us to propose the second contribution, which focuses on the consolidation overhead. We proposed a general method to quantify this overhead and be able to manage its effect on performance degradation. To sum up, this paper improved the management of energy efficiency in datacentres’ servers through the $Ci{S}^2$ index and the server consolidation determination method.