Status update


This is a small update to just let people know what I’ve been up to. This will be a non-technical post.

There are 3 components which  I want to get done before I put any code publicly online.

  • Cabal support (being able to build and run Haskell projects from the IDE)
  • Documentation support (class browser, quickinfo docs and F1 help integration along with jump to definition)
  • Intellisense support

Of the 3 I want to get Cabal support working, then release something, get feedback while I work on the other two. I’m working on all 3 concurrently (mostly depending on which part of visual studio I want to mess with that day) So how far along are they.

  • Cabal: I had a first version which hooked into the Cabal library and using quite a bit of reflection hacks and code changes to the MPF templates managed to load .cabal project files. This first approach was because I wanted to talk directly to Cabal, and not go through any intermediate layers. The reflection hacks were needed because the MPF templates are hardcoded to MSBuild, which uses an XML file format. Not using MPF would mean writing all that code myself which would have taken ages. Unfortunately this only worked sometimes, other times I would get an exception from deep within visual studio. I also had no idea how I would get building to work.

    Ultimately I decided to scrap the entire approach al together, It wasn’t worth the hassle and would be hard to maintain. I’ve now settled on the idea of converting .Cabal files to an internal MSBuild script (and back to .cabal when saving), while adding new templates for a Haskell target type which just invokes cabal-install. This is a much simpler approach which takes some of the burden of maintenance of me and into other tools, but which unfortunately requires me to learn about MSBuild. Currently I’m creating the conversion tool Cabal2MSBuild, which is about 20% done.

  • Documentation support: Documentation from the current module is gained from the AST inside GHC (hopefully, haven’t checked the data I get back yet). Documentation on external modules (e.g. package modules) are gained from two places (hopefully). For quick info the intellisense cache will be used, for class browser (e.g. browing of current project and dependencies) the .hpi files will be used. For F1 help haddock generated documentation will be used.

    However in order for the haddock documentation to be integrated into visual studio it has to be in the correct format. As it turns out, documentation has been greatly simplified in visual studio 2010. The documentation format basically comes down to a zip files renamed to something else, which contains a simple index xml file and just xhtml content files. Great news there, since haddock already does generate xhtml. However I still need to modify the files generated to include a few meta tags, which will be used by the document installer to create indices of the html files, and for F1. I’ve started the modifications to Haddock and they’re about 60% done.

  • Intellisense: I already have the .hpi files, which were those simplified indices of packages. I would like to provide intellisense for both project files and standalone Haskell files. There are a few ways to approach this. From what I’ve seen in the past, visual studio builds a intellisense cache file from the project dependencies on the fly on first launch/use of the project. If you have a large package database this could be handy, it limits the search space, but features such as auto-add imports/dependencies will become harder, as I would have to do 2 checks (local cache, and if not found hit a global cache). Standalone files also require me to only use the bigger (slower) global cache.

    However the speed of that global cache hasn’t been measured yet (because the cache hasn’t been made yet) . So for now, until I have some hard numbers, I’ve settled on just always using the globally constructed index. This is also about 20-30% done. The majority of the work left is reading some documentation and papers.

Hopefully this informs you what I’ve been up to the past few weeks,

Intellisense Part 1–Haskell Package Interface


As most of you who have been following this blog know I have IntelliSense and Cabal support left. I decided to focus on IntelliSense first (even though Cabal support is easier). So this is the first in a series of posts on how I’ve decided to implement IntelliSense.

[Sidenote: University has started again, So I’m afraid I’ll only have time to work on this project in the weekends, at least, when I’m coherent enough to Smile]

IntelliSense for those of you who don’t know is Microsoft’s implementation for Code Completion, a small overview can be found [here]. However the gist of it is that when the user starts typing in a relevant place that the IDE will try and help the user along by showing identifiers and or types currently in scope. To that extend Visual Haskell will support two types of scopes

  • Function scopes: e.g. whenever you’re inside a function, you’ll get a list of every bindings (both local and global) ,lambda variables and Modules in scope. Should you type a module name and a . you’ll get the other module names you can choose or functions you can use qualified from that module if any.
  • Type scopes: e.g. whenever you’re working inside a type signature, the list will limit itself to types that are currently in scope (along with modules again of course).

This is how I plan to implement code completion, If anyone has any requests of suggestions please let me know now since I can still change it for the initial release now.

In order to implement IntelliSense I need to index all the packages currently installed by GHC and also keep updating this as time goes by and you install new cabal packages. Visual Haskell will ship with a custom version of cabal-install ghc-pkg (and eventually a custom haddock as well in order to generate Visual Studio help files) so keeping them up to date should not be a problem.

I have still not decided how to store this information, But I’m leaning towards a structure with a Spatial Index , more specifically I’m leaning towards using a BANG file. I believe using this file will allow me to do the different kinds of lookups I need to do while having a memory mapped file.

But the first step is to get the information from ghc-pkg and ghc on your packages. These are then stored in a .hpi file (haskell package interface). Which is just a very simplified version of the .HI files ghc uses. They contain functions + documentation, classes declarations, instances and types. The reason for these files is two folds:

  • For the class browser we want to be able to browse packages (in a simplified manner) so these files will contain all we need for now, along with the location of the actual .hi file if we need it for more complex stuff later.
  • From these files I will generate the large IntelliSense database, this will not contain any information on classes etc. so we need a way to quickly get to  these. (especially for things like code snippets)

In any case, the first step is now completed, I can successfully generate .hpi files with all the content described above. It does this for my configuration, which contains

C:/ghc/ghc-6.12.1\lib\package.conf.d:
    Cabal-1.8.0.2
    Win32-2.2.0.1
    array-0.3.0.0
    base-3.0.3.2
    base-4.2.0.0
    bin-package-db-0.0.0.0
    bytestring-0.9.1.5
    containers-0.3.0.0
    directory-1.0.1.0
    (dph-base-0.4.0)
    (dph-par-0.4.0)
    (dph-prim-interface-0.4.0)
    (dph-prim-par-0.4.0)
    (dph-prim-seq-0.4.0)
    (dph-seq-0.4.0)
    extensible-exceptions-0.1.1.1
    ffi-1.0
    filepath-1.1.0.3
    ghc-6.12.1
    (ghc-binary-0.5.0.2)
    ghc-prim-0.2.0.0
    haskell98-1.0.1.1
    hpc-0.5.0.4
    integer-gmp-0.2.0.0
    old-locale-1.0.0.2
    old-time-1.0.0.3
    pretty-1.0.1.1
    process-1.0.1.2
    random-1.0.0.2
    rts-1.0
    syb-0.1.0.2
    template-haskell-2.4.0.0
    time-1.1.4
    utf8-string-0.3.4

C:\Users\Phyx\AppData\Roaming\ghc\i386-mingw32-6.12.1\package.conf.d:
    Cabal-1.9.2
    HTTP-4000.0.9
    Hs2lib-0.2.2
    MonadCatchIO-mtl-0.3.0.1
    QuickCheck-2.1.0.3
    ansi-terminal-0.5.3
    binary-0.5.0.2
    colorize-haskell-1.0.1
    cpphs-1.11
    deepseq-1.1.0.0
    fgl-5.4.2.2
    ghc-mtl-1.0.1.0
    ghc-paths-0.1.0.6
    ghc-syb-0.2.0.0
    haddock-2.7.2
    haskell-lexer-1.0
    haskell-src-1.0.1.3
    haskell-src-exts-1.8.2
    haskell-src-exts-1.9.0
    hint-0.3.2.3
    mtl-1.1.0.2
    network-2.2.1.7
    parallel-2.2.0.1
    parsec-2.1.0.1
    primitive-0.3
    tar-0.3.1.0
    uuagc-0.9.10
    uuagc-0.9.14
    uuagc-0.9.23
    uuagc-0.9.26
    uulib-0.9.10
    uulib-0.9.12
    vector-0.6.0.2
    zlib-0.5.2.0

In about 39.16seconds and swallowing about 500mb of ram to do so while maxing out a core. So users will most likely not notice this first step at all. A snap of what the internal of a .hpi file looks like is:

image