Reinforcement Learning for Architectural Combinatorial Optimization

Leon Wietschorke, Yuxi Liu, and Jianpeng Chen¹

1 Digital Design Unit Technische Universität Darmstadt Darmstadt, Germany

Fig. 3: Visualization of the aggregation in rhino and the corresponding observation matrix

Creating an interface for architects to use reinforcement learning algorithms in their usual design software empowers architects and designers to come up with new solutions for old problems. In a capitalist society dealing with a climate crisis it is an interest to find optimized solutions to the problems of floor planning, material distribution and recycling elements. In this report we present an interface that creates a link between the design environment of Grasshopper and the RL algorithms of stable baselines 3.

Fig. 4: Module with connection points and world axis

The floor area of the world needs to be nearly doubled until 2060 and as building and construction today use 36% of the global final energy and produce 39% of the energy-related carbon dioxide, the construction process needs to change drastically. It not only needs a lot of energy but also produces a lot of waste as falsework is often only used once.

We think that robots in combination with AI and an efficient material distribution have the potential to revolutionize the building industry. This IP-Project is a first step into solving architectural combinatorial problems with the help of reinforcement learning (RL).

Fig. 5: A SL-Block, a conjugate pair and six types of engagements.

The method of reinforcement learning can be one option to solve complex design problems as it reacts to the environment. This report presents a general interface between the architectural design software of Rhino and Grasshopper and the scripting environment to run RL algorithms. In three case studies we show different approaches on how this interface can be used to solve combinatorial problems. As this is an early research on implementing RL algorithm in an design environment for architects it lays a stepstone for future research.

Fig. 6: Voxelized form that should be rebuild by a sequence of six engagements

Rhino is a 3D modeling software that allows users to create complex geometries. With Rhino, however, only a direct modeling is possible. Only with the built-in plug-in Grasshopper a parametric modeling is possible. Grasshopper is a visual programming language that uses connectable components to create 3D geometry.

It is thereby a very powerful tool for architects and designers, because it allows to parametrically change the geometry without the need of totally rebuilding the object. In 2007, where Grasshopper was published for the first time, this was quite a novum in CAD applications and still today the parametrization of buildings is rarely done.

Fig. 7: Different states during training

Various projects show the potential of parametrized buildings. It not only provides a huge freedom of design but can also directly generate machine code for robots and CNC machines to manufacture the buildings. This opens up the question why not more projects a thought form design to production. Grasshopper provides an open platform that allows third party developers to create their own plugins for it. This enables us to implement RL algorithms in Grasshopper.

Fig. 8: Process of PPO reinforcement learning algorithm by using Grasshopper components and the socket connection

The aim in this case study is to rebuild given shapes with a sequence of SL-Blocks. As there are many ways to connect just two SL-Blocksthe combinations for a sequence of blocks get so high that trying out every possible sequence, to find the best, would take too long.

Therefore the algorithm should learn a policy on how to combine SL-Blocks to quickly rebuild a certain shape. To measure whether a shape is rebuild accurately, the goal form gets voxelized. The agent sends an action to the environment, in this case one number, that represents one of the conjugate pairs.

Fig. 9: Compression only structures and stock set of elements

The environment then tries to engage this new conjugate pair to the already existing structure. The engagements are added sequentially forming a string of conjugate pairs. The result of the learning process should be a sequence of SL-Blocks encoded in a string of conjugate pairs, that rebuilds the goal form in the best way.

Fig. 10: Fitness evaluation of connection points

Goal of the optimization is to select the optimal subset of stock elements to rebuild the goal structure as good as possible. Thereby some critical elements need to be as exact as possible. This leads to a prioritizing of elements in the structure. As there is a fixed amount of elements that can only connect at their ends, the action space is discrete. In combination with a high quantity of elements this leads to a high amount of combinations.

Standard algorithm is not optimized for solving discrete problems. In order to get a solution to this problem we transfer the problem into an assignment problem. Therefore, the elements are defined as agents and the line segments in the structure are the tasks. This enables us to use other techniques to solve this problem.