Building a High-Performance Binary Serialization Format with In-Place Modification - Hamish Morrison

YouTube Excerpt: https://cppcon.org --- Building a High-Performance Binary Serialization Format with In-Place Modification - Hamish Morrison - CppCon 2025 --- There are a huge number of binary serialization formats out there: protobufs, Thrift, Avro, MessagePack, flatbuffers, BSON, CBOR, and many more. How do they differ? Which one should you use? How fast are they? Codegen vs non-codegen? Schema-based or schemaless? Zero-copy or not? Within our systems we use many different serialization formats for encoding market data. These formats range from the simple, to more complex tagged formats with compression dictionaries. We recently needed an efficient schema-less data format with support for arrays and maps, and -- critically -- the ability to mutate the data quickly in-place, without re-encoding the whole buffer every time. Essentially, a compact, binary version of JSON, with in-place modification. This talk walks through the design and implementation of this binary serialization format. We'll cover: - Creating a simple binary serialization using tags and values - Implementing arrays using offset tables and indirection - Using SIMD for fast searches in offset tables - Building more complex data structures in contiguous buffers, in order to implement maps - Minimizing copies during in-place updates - Benchmarking against other serialization formats and standard library containers - Exploring the trade-offs of our format Throughout the talk we will be developing an ergonomic C++ interface for the serialization format, allowing users to interact with the encoded data similarly to how they would interact with ordinary vectors and maps. While the focus is on serialization, the talk should be interesting to anyone who is interested in data structures and high-performance computing. --- Slides: https://github.com/CppCon/CppCon2025/blob/main/Presentations/Building_a_High-Performance_Binary_Serialization_Format_with_In-Place_Modification.pdf Work at Hudson River Trading (HRT): https://tinyurl.com/safxfctf --- Hamish Morrison Hamish Morrison is a software engineer at Bloomberg where he works on high performance market data processing systems. --- CppCon is the annual, week-long face-to-face gathering for the entire C++ community. The conference is organized by the C++ community for the community. You will enjoy inspirational talks and a friendly atmosphere designed to help attendees learn from each other, meet interesting people, and generally have a stimulating experience. Taking place this year in Aurora, Colorado, near the Denver airport, and including multiple diverse tracks, the conference will appeal to anyone from C++ novices to experts. Annual CppCon Conference - https://www.cppcon.org https://www.linkedin.com/company/cppcon https://x.com/cppcon https://www.facebook.com/CppConference https://www.reddit.com/r/cppcon/ https://mastodon.social/@CppCon --- Videos Filmed & Edited by Bash Films: http://www.BashFilms.com YouTube Channel Managed by Digital Medium Ltd: https://events.digital-medium.co.uk --- #cpp #cplusplus #programming #coding #softwareengineering

https://cppcon.org --- Building a High-Performance Binary Serialization Format with In-Place Modification - Hamish Morrison - CppCon...

Read Full Article 🔍

Curious about Building A High-Performance Binary Serialization Format With In-Place Modification - Hamish Morrison's Color? Explore detailed estimates, income sources, and financial insights that reveal the full picture of their profile.

Source ID: pADE-7xfAH0

Category: color style guide