Nov 25 2013
Lately I'm starting to learn a bit of how node.js works, so after some fights I've finally been able to have a decent setup for development without having to rely on a remote *NIX machine running node.
I'm just assuming that you already have installed Visual Studio 2013. I have the Premium Edition, I don't know if with the free Express Edition everything will work (specially node tools for VS).
This are the steps to setup most basic stuff needed:
1) Install node.js
I used the MSI installer x64 without any issue.
2) Install Pyton 2.7.x
Seems that 3.x is not fully supported, so I installed 2.7.6 (64-bit).
Remember to add to the PATH environment variable it's path, it didn't on my installation and node-gyp was not working.
3) Install node-gyp
Required for modules that need building from their sources. Uses Python and under windows Visual C++ runtime builder for .node files.
4) Install Node.js Tools for Visual Studio
Really useful, not only provides with project templates for node apps and websites, but also allows cool things like local or remote debugging or intellisense for all code.
This VS Tools are in alpha, but the main problem you will encounter with it is that npm will fail a lot, as many modules have only been tested under Mac/Linux and use .sh scripts, that of course won't work under Windows.
5) Install nodemon
To avoid manually restarting node server, I'm lazy so for me a must-have
Installing modules is the only painful part, but going to a Node command prompt and manually "converting" to windows .sh commands and running "node-gyp configure build install" usually either works or pops out required dependencies that you should fetch manually first. It's painful but blame the module developers, seems that node and npm properly support now Windows but only some up to date modules seem to take this OS into account.
Oh, and if after building node-gyp fails not finding the .node "binary", go to the \build subfolder, probably VC++ created a \build\release subfolder and put it there.
Oct 24 2013
Back in 2008 I was working at NAVTEQ, who had just been bought by Nokia, in the Madrid's R&D offices. There, the coolest project I worked at by far was optimizing an internal tool used to create indoor routing paths from grayscale images of interior maps.
It had lots of cool features, but I focused on trying to make it's A* search algorithm and heuristics as fast as possible, to try to evolve the application into something usable in realtime, for example as a web service, and scalable to theorically hundreds of concurrent users.
It was going to be field tested at venues, so I went for hardcore scenarios like labyrinths instead of boring (and easy) corridors and square rooms.
This is a small summary of how it worked and how did I optimized it, based on the notes I took back then and the slides I prepared when I presented the project. Plus remembering as much as I could about A*, which I haven't touched again in +5 years. So bear with me if I make some mistake here or there...
The initial routing (already done)
1) Create the map mesh
Differentiate between walls and floor, or walkable/non-walkable tiles.
The mesh resolution is quite important to avoid innacuracies:
Which can lead even to errors by "walking through walls" if the resolution is too small:
2) Calculate costs mesh
Assign each walkable tile a cost value. Based on its distance from the walls, the further, the higher the cost, because an algorithm wants to travel as less tiles/spaces as possible.
3) Run the A* search
Then we can finally run the A* and let it map a route. Note that line simplifications and edge/turn detections are performed to obtain a more realistic navigation line drawing.
We'll get into what those yellow points mean right now. Let's just say that the default A* algorithm used them sometimes as "desirable positions" but as they cost a lot, if the heuristics found a faster route would tend to ignore them. The tool had implemented 3 different heuristics for short-circuiting the A* with a good enough solution, results would vary depending on which was used.
This approach works and gets nice results, but it consumes a lot of CPU and needs to keep thousands of points in the mesh. Even counting with storing a pre-calculated points mesh and cost mesh for each map it wasn't fast enough.
1) Mark cost mesh maximas (already done)
Both due to A* trying to find the shortest path and due to our previously calculated cost mesh, if we run the A* right now, it would calculate the shortest route... but walking as near to the wall as possible.
Neither human beings usually walk touching the wall nor we want visually to look "so optimized", so instead, we will mark for the A* only the local maximas, those points inside a given radius that cost most (because are placed in the middle of corridors, rooms, etcetera and are less optimal for fastest paths).
This always has a maximum radius of calculation, so in huge rooms if the radius is not enough you might miss some points.
Maximas were already calculated to work as "attractors" for the algorithm when searching for the path, to try sometimes to make the line stick to the middle instead of going near the wall, but nothing else.
2) Simplify the map to only use maximas
Go from almost 200k nodes...
To just ~300 keeping only the maximas, before any further optimization:
3) Simplify the maximas and use them as navigation nodes
We can apply a line simplification algorithm and edge/turn detections (if angle is greater than x degrees from current direction vector, consider it a turn/edge and do not remove that point) to get rid of many maximas.
And if we then apply a line of sight optimization... we can reduce even more!
Storing just 4 nodes for two corridors and a 180 degree turn is really nice (points used are those on edges of blue lines).
As the resulting line would be a bit ugly for a GPS-like representation, we could still store the pre-LOS nodes list to smooth the line into splines if we wanted to do so. After all what matters here is to calculate the A*, not the actual drawing of the line, which would be done client-side.
Having so few points was quite trivial to apply a Diagonal Distance heuristic and have a really fast A* applied in realtime.
Extra) Cache the costs
This might sound trivial, but whenever you can, cache the cost mesh, calculate it once and do not recalculate until needed. And then, try to only recalculate affected areas.
This is useful for example in a tower defense game, where you only need to recalculate enemies path whenever you place a new "soldier", or an RPG where you are moving inside a map that it's fixed and only needs to calculate a new path when the player clicks in a new position or object.
Even before the optimizations, having binary serialized meshes with precalculations instead of on-demand map processing saved seconds of initial load.
Thinking about this optimizations and actually making the application perform them (including tests!) has been of the most interesting projects I've done.
I don't know if this, along with other projects, were used in some indoor mapping Nokia software, but there were some other really cool ideas and PoCs when our Madrid R&D office was shut down.
Sep 19 2013
12 years ago I was studying at the university. It was my second year there and despite I thought I knew a lot, now I see I was still quite newbie in many areas.
We used to use floppy disks for our assignments, booking the laboratories or shared computer rooms. We spent hours waiting to get previous years exams, samples and sometimes just theory chapters, all in paper. We used to have everything installed on our PC: Pascal, C, C++, Assembler, Java... We used to wait for teachers to go to their assigned mentorship hours.
It worked (mostly, as I quit my studies, but not due to this reasons), but was clumbersome, even when I bought my first laptop to make better use of free time gaps. After all, I had between 45 minutes and 1.5 hours of travel to there (+ another 45 min of coming back home).
Now, fast forward to the present, and here I am, trying to slowly but surely finishing the studies.
Some things haven't changed so much, but man, others have a lot...
- My main study "book" now is a tablet: Easier to bring with me even on weekends if I go outside of the city, comfortable to read (retina-like display is now "mandatory" for me), and with 3G I can easily check internet without problems. If I need a book I can have thousands in PDF or EPUB available.
- Laptops are way more powerful and lightweight: No more carrying a 3Kg, 15" laptop plus a big charger. No more long delays due to compiling... My new ultra-thin laptop weights ~1Kg, battery lasts +8h and I'm in love with Intel i7 CPUs and how well they perform.
- I virtualize everything: Do I need to use Java and Eclipse for this course? No problem, VM cloning and I'm ready to install inside whatever the teacher wants. As if I had to install an MS-DOS or command-line Linux. With my recent laptop adquisition and its 8GB of RAM this is heaven, as I can have Java VMs with 2GB of assigned RAM.
- I communicate with the teachers via email and/or forums: This is one of the few advantages of studying remotely and not in a "normal" university, but once you test it, you never want to go back to physical teacher hunting.
- I am more organized: I have a "studies" Google Calendar, I do a weekly planning of how will I approach the studies, I try to finish even early all assignments and exercises to have time in case of problems. I am no longer postponing boring tasks to the last weekend.
- I am faster (sometimes): I see more clearly where to focus my effort, what I'm being asked for, and I don't waste time in unwanted "decorations", unwanted features or incorrect paths of action. This is mostly experience but... is still an improvement.
- Tools have improved and are more available: We no longer need to have pirate software like Windows NT, Rational Rose or some paid Java IDE. Microsoft DreamSpark, ArgoUML, and NetBeans are way better free solutions, just to give some examples. Legal, better, faster, more advanced, full of features...
- There is a flood of information: I built La Web de Programación in 2001 to group and make easily available all (non-coprighted) books and tutorials I found, because was hard to find decent and varied development resources. Now, between StackOverflow, Wikipedia and may other resource sites, we have more to read and learn than we will be able over our whole life.
- I value my time more: I spent two years studying and only working in the summer before doing part-time job in the morning, study in the afternoon. I did my coding in the gaps, didn't wasted time at the cafeteria playing cards... But I also played lots of videogames, went out partying too much and in general wasted some time that could had saved me from failing two or three courses (and I would have less to finish now!).
Now when I say I study, every minute spent has to be worth it because I don't have too many available.
- I enjoy learning every course *: Some of them will be surely hard and exhausting, but my point of view now is more of a personal challenge instead of a parental and social pressure ("you're nothing without finished studies"). You can walk everyday with a smile or angry instead; You can always choose, and your decision fully changes your perspective, despite it doesn't change the world itself.
Times change... and not always for worse!
* I used to become frustrated of the lack of passion in many of my teachers, against my thirst for learning more and code bigger and better programs than what I was assigned/asked for. While this doesn't exempt them from being bad at their work (teaching), I now see that that fact shouldn't have stopped me from studying; I should have kept fighting to finish and demonstrate to them I could be a passionate engineer despite their futility.
Sep 14 2013
The free edition contains the editors, but sadly no integration with source code control systems, code formatting, inspectors or code snippets. The Lite version costs 40$ and contains everything except the framework plugins (supports a good list of FWs like CakePHP, JQuery, Joomla or Wordpress), which can be bought separately or directly purchasing the Pro version (for 120$).
I've been given a Professional license so that's the edition I'm going to talk about.
It mostly works as any other IDE you might think... Visual Studio, Eclipse or anything similar. The look & feel is normal, no fancy graphics, no flat UI nor special buttons, just what it needs to have.
Feels fast and quick responsive to me, autocomplete worked quite nicely with a pair of projects I opened with it and in one it even detected a typo inside a file :)
Typical (customizable) code coloring, folding, searching and highlighting works great too.
Configuration options are more limited that I'm used to, many of them are still the relevant ones but feels more basic than competition. I wasn't able to configure an FTP connection so I'm not sure for example if I can use SFTP or just the basic one.
Code completion seems to work ok but again here other competitors win, doesn't hints PHP variable types, you cannot type foreach variables, typed function parameters don't have a hint either... It is way better than nothing, but not perfect.
About the plugins, as I'm currently not doing anything with any of the supported frameworks I checked the official website to see what features do they have. Seems that each one contains not just syntax autocomplete and suggestions but also project templates and custom windows (like a Symfony tasks executor or a Wordpress Theme Editor).
Of course it features also a debugger if you have a local install of PHP. As I have just switched my PC for a new one I still haven't configured the VMs so couldn't test this, but seems to have the typical breakpoints, pausing, step into/over...
The free edition is quite nice featured for being fully free. The Lite version gets even more interesting, as it really is cheap and contains most features you would want for a decent IDE. As for the Professional version... I'm in love with PHPStorm and still think has no sutiable competitor, so CodeLobster feels inferior and more limited, although is also cheaper. If you use one of the frameworks maybe gets more interesting, for me the best "deal" looks like the Lite version.
Not a bad IDE but needs more features to fight with the big guys in the Pro area.