This paper reports on our progress and findings in building a Web annotation system for non-immersive 3D virtual environments. Over the last two years, we developed and tested two systems for collaborating designers to comment on virtual 3D models. Our first system, Redliner lets design team members browse and leave text annotations on surfaces in three-dimensional models. Experience with Redliner, including two user evaluations in different settings, led us to develop Space Pen, a second annotation system with improved interaction capabilities. It goes beyond the post-it note metaphor, allowing users to draw in and on the virtual environment.