This paper demonstrates an experience in the development of a design performance evaluation system that can frequently evaluate building design performance within the design process in a real-time manner. The evaluation system, that we call "Vitruvius Studio," is composed of several modules such as a front-end component-based CAD engine, a knowledge base, and a set of design agents. While a computer programmer compiles a set of programming codes to check compiling errors during the implementation of a software system, an architectural designer can 'compile' his/her intermediate design product to evaluate design errors during the design process. The compilation can be done immediately at any level or any time during the design process in a real-time manner. We expect that this compiling process will dramatically increase design feedbacks, and thus result in a better design product.