Hey Unity Folks,
I’m pretty sure you are all familiar with Script Inspector 2, and many of you are already using it for viewing and editing your scripts, shaders, and text files right inside Unity Editor, so I’ll skip the introduction and just present what is its current WIP status here.
For the last couple of months I’ve been working on automatic code completion feature, also known as “Intellisense” as Microsoft likes to call that. At first sight the feature may look not too complex, but as it turned out it is complex as hell if you try to do it all from scratch without any help of some of the few ready made libraries. One of those is NRefactory, an open-source library used in MonoDevelop/Xamarin for example, and another one is Roslyn by Microsoft which is still not finished and available as a preview for now. Both libraries can “understand” only C# code which was the main reason I’ve decided to go with my own solution since Script Inspector 2 besides C# scripts works also with UnityScript/JavaScript, Boo, and ShaderLab. The second reason was that I wanted to fully understand the mechanics behind the algorithms used in the library so I can easily fix bugs or extend the functionality when needed.
So, I did lots of research about the techniques and algorithms needed for implementing such feature before I started working on it, and at that time it looked like I might be able to invent quickly a minimalistic solution that will give good enough results… However, after trying a couple of methods and as I was getting deeper into the topic I realised I was wrong and that I’d still have to implement all the processing phases, almost same as it is done in NRefactory and Roslyn. That discovery was a little disappointing but it didn’t discourage me to finish my job.
I begun again implementing the feature in the proper way, as it should be. First it needs a parser (a proper one, not just some hack with regular expressions) that will be able to recognize the syntax of the source file, generate a parse tree, and then incrementally update the parse tree while the user is editing previously parsed source code. Next step is the semantic analysis which traverses the parse tree and extracts symbol definitions, builds symbol tables, tries to resolve symbol references, etc… Once symbol tables are built and symbol references resolved, code-completion is simple: it takes the symbol to the left of the cursor, finds its definition in the symbol table, reads its type from the definition, and then list all the members from that type which are accessible in the current scope…
Sounds simple, eh? Well, there’s still lots more than that… The parser needs to gracefully pass over syntactically incorrect input (there will be a lot of that as you type), and has to be able to deal with contextual keywords, for example. The semantic analysis has to deal with complex language features such as type inheritance, overloaded methods, type inference and generics, which is all needed for correct expression type evaluation… All that turned out to require tremendous amount of not only work, but also deep knowledge of all those topics! :-?
It took me more than a month to find all the related academic works and to study them, and then even more to implement some of them and to adapt the algorithms to work with the rest of Script Inspector 2 code. And still, it isn’t finished… So far I’ve built a blasting fast parser with support for incremental parsing and pluggable LL(*) grammar definitions (so that I can reuse the same code for different scripting languages), C# 3.5 grammar with support for contextual keywords, syntax errors detection and recovery, syntax error highlighting in the editor, a little bit of the semantic analysis, and a little bit of the semantic highlighting shown in this screenshot:
The screenshot also shows proper handling of some of the contextual keywords in C#… Well, this may look like not too much, but in fact there’s much more than a screenshot can show! I’ll have to polish the error highlighting a little bit more and I’ll post a video showing that in action! Stay tuned for updates