Why the Construction Industry Needs a New Open Format

Summary

• Construction, the world’s largest industry is less productive today than it was in 1968
• ‘Walled Gardens’ of software makes a few rich but is detrimental to most AECO professionals and workflows
• Existing 3D formats are relics that don’t care about AECO’s unique needs, and can’t deliver at scale in real-time
• VIM has architected a modern, efficient, open specification 3D format that powers the next generation of AECO tools and experiences - giving professionals control to make better decisions faster

The Problem

Fractured processes, worsening collaboration, and a leaner workforce has reversed construction labor productivity over the last 70 years while the rest of the world’s top industries get better. Architects, engineers, contractors, and owners (AECO professionals) recognize the problems and know that software technology can help fill the gaps, but they are far from realizing the full potential benefits of software technology.

The first step in improving this situation is for the industry to start demanding a more modern, efficient, and open format for the interchange of BIM data. Our proposal to address this is the VIM file format.

Why is AECO Software Tech Lagging Behind?

The construction industry is saddled with 20-year software and data-formats that are unable to leverage the processing and graphics capability of modern computing devices. Larger projects might never be loaded all at once, and basic processing tasks can take anywhere from minutes to hours.

There have been many hardware advances in the last 20 years that have fundamentally changed how data and software must be structured to reach the full potential of today’s computers:

• Graphical Processing Units (GPUs), which are dedicated hardware for rendering millions of polygons, are now ubiquitous.
• Parallel computing is now the norm, not the exception, even mid-tier smartphones now have multiple processors.
• Collaboration is the default - people expect data to be hosted remotely and accessible everywhere while maintaining privacy and security.
• Computing platforms vary widely in terms of capability and function. These range from smart-phones, wearable devices, tablets, laptops, desktop machines, to remotely hosted super-computers.

Today a lot of valuable digital construction data (BIM data and beyond) is locked up in closed proprietary data formats which are bound to aging software packages developed by a single software vendor. As a result, stakeholders in the construction process (e.g. contractors, sub-contractors, estimators, specification writers, owners, facility managers, etc.) are unable to fully leverage the BIM data to make better decisions or optimize their processes. Some of the negative results are miscommunication and misunderstanding between stakeholders, inaccurate estimation, change orders, scheduling and planning conflicts, errors in constructability, and so on.

Two decades ago, it was common practice for data formats to be only read and written by a single piece of software, on a single platform. This is no longer adequate.

As the internet has grown in importance, and as computing devices have become more specialized, well-designed open data exchange formats such as PDF, HTML, JPEG, MP3, MP4, and XML have become crucial to the effective exchange of data.

The Solution is Open

So what is an open format, why is it good for the construction industry? First, according to Wikipedia: An open format is a file format for storing digital data, defined by a published specification usually maintained by a standards organization, and which can be used and implemented by anyone. For example, an open format can be implemented by both proprietary and free and open-source software, using the typical software licenses used by each. In contrast to open formats, closed formats are considered trade secrets.

Open Formats are Good for Customers

As a customer, it is risky to build a dependency on technology based on a single vendor’s closed format. Users place themselves entirely at the mercy of one business: their pricing, tooling, and support. This has proven to be a real problem for customers that have their data locked up behind a walled garden that they must pay exorbitant amounts of money to have limited access to it. This is a relic of an older business model from the time when most computing was done in isolation on a single machine. This model is extremely outdated. Customers expect to be able to have contextual views of the same data across different platforms with varying capabilities.

Open Formats are Good for Businesses

Technology that is open is better. When a community of experts can examine and identify inconsistencies or ambiguities in a specification, it assures that the product is of better quality. As a result, open formats have more longevity, and will often evolve into something better.

Another benefit for us at VIM to develop an open format is that it encourages us internally to keep our own house clean. Knowing that everything we do is potentially up for scrutiny and has ramification for others. It encourages better software development practices.

Evaluating Existing Formats

At VIM we started our journey with a business requirement to bring BIM models and data quickly into lower power Virtual Reality and Augmented Reality devices. We carefully evaluated many different file formats before starting our journey. The four primary contenders for 3D BIM interchange into real-time which we evaluated were: FBX, glTF, USD, and IFC.

FBX (FilmBox File Format) is a widely used file format originally from Kaydara and is now owned and maintained by Autodesk. It has become the default asset transmission format in the games industry. Even though FBX has a free SDK, it is not open format. Reading and writing FBX files requires the binary SDK and it cannot be ported to web-browsers. FBX was originally designed for animation data. It does not do a good job of conveying the semantics of BIM data and is inefficient at handling large geometry.

glTF (GL Transmission Format) is an open data exchange format for 3D geometry and animation data that was designed specifically for WebGL. The glTF format is a difficult specification to fully support (for reading and writing) and requires custom extensions to support BIM data and geometry data channels beyond a limited initial set.

USD (Universal Scene Description) is a framework for authoring and rendering 3D scenes composed from multiple sources. There is no formal specification for it as a format and is instead defined by its implementation.

IFC (Industry Foundation Classes) set of specifications and libraries for supporting the interchange of construction data. IFC was started in 1994 by a consortium of 12 companies led by Autodesk.

IFC was designed to express parametric surfaces and solids and their associated BIM data. This means that to make a rendering, the parametric surfaces and solids must go through a very lengthy conversion process into a triangular representation appropriate for rendering by the GPU (Graphical Processing Unit). The GPU specializes in drawing lists of triangles very quickly. The conversion of a parametric surface to a triangular format is called “triangulation” or “tessellation”.

The process of triangulating complex intersecting parametric surfaces is very time consuming for a computer, and hard to do correctly. It requires a specialized library, called a geometry kernel. With the goal of reducing the time required to display the geometry, this made IFC unacceptably inefficient for us.

Our Findings Left Us Searching for More Answers

We found that these formats required too much complexity to parse large scale BIM projects correctly and efficiently on low-power devices, and they had no built-in mechanism to carry the structured BIM data. As such, a new solution was required to accommodate the specialized needs of the BIM industry.

Designing An Open Format Built for the Future

We created VIM as a file format tailor-made to address the challenging performance requirements of large BIM projects on a variety of devices. We have been evolving the VIM format over the last couple of years, testing it on hundreds of large real-world projects, on multiple platforms (desktop, mobile, VR/AR, and web browsers), and in libraries written in multiple programming languages (C#, C++, and JavaScript).

A Set of Principles Guided Our Vision

While designing the VIM format to satisfy the rigorous requirements of real-world construction projects, we strive to follow these principles:

• Do not try to be everything to everyone – do one thing and do it well.
• Minimize the amount of processing that clients must perform on deserialization.
• Transmit geometry data in a GPU friendly format.
• Retain the relational nature of the BIM data structure.
• Do not tie the format to a particular hardware, operating system, or software.
• Make the format straightforward and unambiguous to parse and validate.
• Test the specification with actual implementations on actual large data at scale.
• Every component of the specification should be simple, self-contained, and not likely to change.
• Compression should not be a feature of the file format.

What Makes the VIM Format Different from All the Rest?

At the top-level, the VIM container format uses the BFAST data format. BFAST stands for Binary Format for Array Serialization and Transmission. A BFAST is a very simple cross-platform binary format which allows data buffers to be associated with names and accessed quickly through a lookup table. By nesting BFASTs within BFASTs, we can emulate a simple virtual file system, like within a ZIP.

A VIM file has six BFAST buffers:

1. Header – easily parsed version information
2. Entities – relational data tables to store BIM data
3. Geometry – all mesh data as concatenated sub-meshes
4. Nodes – a scene graph
5. Assets – associated data files like textures, and renders
6. Strings – a string table so that strings can be indexed and deduplicated

VIM geometry data is broken up into a scene graph (nodes buffer) referencing mesh data (geometry buffer) stored as using the a G3D data format. G3D is a low-level high-performance representation of 3D geometric data. The VIM G3D layout consists of standard data buffers like vertices, indices, UVs, and materials, as well as indices indicating the beginning and ending of each sub-mesh.

BIM data is stored as a collection of relational data tables in the entities buffer. Each data table has several columns which are either numerical, strings or are keys referencing a row in another column. BIM properties are stored as a collection of key/value pairs.

For more information and details of the specification, we direct you to our GitHub. If you are interested in our libraries for reading, manipulation, and creating VIMs please contact us.

Where Do We Go From Here?

Here’s where we need to go now. It’s also what our software partners are doing and what they want us to share with the AECO software community:

1. All AECO processes can improve, and software is a great catalyst for that change.
2. Open is the future, no one wants to subscribe to a closed software ‘walled garden’ and be stuck forever.
3. VIM, it turns out, has the right formula for real-time performance + project scalability + open access to deliver the promise of the next generation of BIM

Wow, it’s been a busy and rewarding 18 months. We love working with our partners and customers and seeing their software performance take off thanks to VIM. Their customers are pretty happy about it, too!