Designers draw diagrams to think about architectural concepts and design concerns. We are interested in programming a computer to recognize and interpret design diagrams to deliver appropriate tools for the design task at hand. We conducted empirical studies to find out if designers share drawing conventions when designing. In this paper we first discuss reasons to investigate design diagrams. Then we describe our experiment on diagramming for designing an architect's office. The experiment results show that designers use different diagramming conventions when thinking about different design concerns. We discuss and report our efforts to implement a freehand drawing program.