javascript.reverse()

Short update of what the JS team is at

We actually wanted to enabled Incremental GC on Nightly, but again we had some fallout and it had to be backed out again. Bill thinks it should reland at the end of the week.

We are happy to welcome Benjamin Peterson, who is going to join us this summer as an intern working on SpiderMonkey’s ES6 support. Benjamin is an active python contributor. He has already started implementing rest parameters.

Till Schneidereit, (a fellow German, finally!) started picking up some GC related bugs, thank you and feel welcome.

In an effort to reduce the memory usage of average JavaScript applications (MemShrink \o/), we came to the conclusion that it is okay to throw away JIT code compiled by Jäger on every Garbage Collection run. Unfortunately this doesn’t work very well for animation heavy scripts like games, where recompiling would introduce long pauses. Brian fixed that.

Jason showed us how to use the new Debugger API to debug JavaScript code running in Firefox.

David Mandelin and me blogged about the SpiderMonkey API (JSAPI), and what needs to change, C++ yeah!

The DataView object landed, thanks to the work of Steve.

Luke just finished a patch that is going to speed up the handling of some function parameters/variables. Besides blocking more IonMonkey performance improvements, it already showed 10% better scores on the v8 early-boyer benchmark. (Bug 659577)

Jan has been working on chunked compilation which should help IonMonkey with very large scripts. But because this is a very broad change and the Ion team likes to focus on stabilizing, fixing crashes and test failures first, this is going to land after the initial release. Luckily these kind of large scripts are uncommon for normal JavaScript, but they are often found in Emscripten compiled code. JägerMonkey (+TI) which has chunked compilation is still going to help those scripts.

Edit: Republished because of some tumblr problems.

How the bad JSAPI is hurting us

The JSAPI has been relatively stable since it’s incarnation together with the first version of JavaScript in 1998. At least till 2008 when we started doing some rather deep changes with the introduction of compartments, the removal of threads and more. In the most cases for more safety, speed and less memory usage. But you could probably still understand code written in that dark ages.

And in 1998, it probably only made sense to use C and thus also design a C API. So and that’s what we are left with today. Luckily not for too long anymore!

It’s time for a new API.

I agree to this sentiment wholeheartedly.

So let me enumerate two recent examples where a better and safer API could have stopped us from introducing bugs.

Case 1: off-by-one errors in JSClass initialization - Bug 747617

Every JavaScript object has a JSClass, this class describes how the object should behave when used in certain situations. It also holds some bits of information that are the same for every JavaScript object of that class. For example the class name, which you can get in JS by using Object.prototype.toString.call. But the main purpose is to implement special behavior for these object, through the usage of “hook”s. For example when you have a bit of experience with the ES5 specification, that’s what we use to implement pseudo internal methods like [[Construct]] (sort of at least). Because we obviously can’t use C++ virtual functions we are left with functions pointers in a struct. Because we can’t really enforce strict initialization rules and type checking failed, we actually assigned some the wrong function (or NULL) to some hooks after removing one function from the struct and not updating all uses in the correct way. This broke an old hack to support document.all("elementID") from ancient IE.

Case 2: JSVAL_IS_OBJECT - Bug 752226

So when I told you that JSVAL_IS_STRING checks if a Value (our way to represent the different types in JavaScript: Undefined, Null, Boolean, String, Number, and Object) is a String, and JSVAL_IS_NULL checks if a Value is null. What would you think is the purpose of JSVAL_IS_OBJECT? If you thought it’s checking if a Value represents an object you were right, but this is only one part of the answer. It also returns true when you actually passed a Null primitive to it. D’oh. This is kind of similar to the typeof operator, which also returns “object” for null and objects. This could have been a mistake, because Brendan already used that API in the wrong way when implementing JS. (Here is the implementation of typeof in Mozilla 1.0). So lets get to actual issue here. After you checked if something is an object it’s obviously okay to use JSVAL_TO_OBJECT and just do what ever you want with it. (JSVAL_TO_OBJECT is used to get a JSObject* out of the Value, like the name suggest this represent an actual object in JS). Of course a lot of people forget that this API returns a null pointer when you use it with a null primitive. When removing all the uses of JSVAL_IS_OBJECT, I noticed at least 5 places where this check was missing and would lead to null pointer dereference crashs. The correct way to check if something is an object would have been !JSVAL_IS_PRIMITIVE, obvious right?!

There are probably a lot of other cases of bugs that were introduced because of our bad API. I can think of at least three more, which often showed that the most developers using the API, don’t really know what they were doing, but can you blame them with functions like JSVAL_IS_OBJECT?

Here is a little gem Wes Garland shared with me that shows the bad zen of the JSAPI.

No, your method does not return false, you are simply making the JS engine stop dead in it’s tracks. If you want your JS function to return true, you need to make its implementation return JS_TRUE while setting *rvalp to JSVAL_FALSE.

Mozilla JS Holiday Update

Although we had holidays, things went smooth, and we had about 130 code landings since 5th December. (Depending on how good my result of mapping commits to bugs worked). This doesn’t include patches landing on the IonMonkey branch, which have steadily grown.

We still find some unnecessary code after the removal of the Tracemonkey. See the dependencies of Bug 698201.

Terrence Cole has been working on adding write barriers, which are required for a Generational GC to work.

Brain Hackett introduced an analysis to find objects that are not explicitly rooted (Bug 707049). We currently use a conservative stack scanner, which means we don’t garbage collect stack values that point to objects. But a conservative stack scanner doesn’t know if it’s actually just a integer, which happens to have a value to corresponds to an object address, or if it’s really a pointer to that object. So we can sometimes keep things alive longer than necessary. 

For a moving GC, we need to identify all roots of an object (every pointer in the program that points to a particular object), so we can move the object around in memory, and rewrite the pointer.

Brain Hackett also worked on introducing inline caching to IonMonkey. We a had quite a problems few with the inline caches in JägerMonkey on Amd64, that we are going to avoid this time :)

Bill McCloskey landed the Incremental Garbage Collector on the larch project branch, from my understanding he is now actually nearly done, but still needs to fix some stability issues.

Jeff Walden continued working on splitting the storage of properties and elements in every object (Bug 586842). After this work has been done, index access on every object is going to be as fast as on arrays. A typical element access looks like, this obj[123]. This is a lot of work because, he does the conceptual splitting of accessing an element or property already at the lowest level, so we don’t introduce bugs, and have a clean cut. Clean-ups in the parser, yummy.

Jan de Mooij has been fixing a lot of bugs in IonMonkey that made us crash when running the v8 or sunspider benchmark. Some of these bugs he fixed last week were in the Linear Scan Register Allocator.

David Anderson already started simplifying the IonMonkey design (sic!), as some unnecessary complexity has been identified. Like Jan he also fixed bugs/crashes in IonMonkey.

Nicolas B. Pierron and Eddy Bruel also have been eagerly working on Ion, implementing new operations (like multiplication, the this operator etc.) and fixing bugs.

IonMonkey now can already run some tests, but most of the time we still fallback into the interpreter.

Luke Wagner was responsible for the code cleanups mentioned in the beginning.

The mysterious Ms2ger is earning a gold star for removing includes of private SpiderMonkey headers from the rest of the mozilla source.

I have been working on some correctness issue and one performance regression from the tracer removal, because we don’t optimize Math.round in the same way anymore. I am also trying to convince everyone the remove E4X.

I hope, I didn’t forget anyone, and you didn’t have too much pain reading until here.

How to use scan-build to analyze SpiderMonkey

Today I finally found the time and engagement to analyze SpiderMonkey with the static analyzer scan-build provided by Clang. Obviously you are going to need Clang, but because it uses gcc! for building the actual source, you need that, too.

I assume you already have gcc installed. Then you need to follow the build instructions for clang. (I think somebody already documented this a bit clearer, but I can’t find that at the moment). Make sure you actually added clang to your path.

Follow the scan-build installation. Usually you just need to add the directory in the clang source ($SRC)/tools/clang/tools/scan-build to your path.

Then go to your js/src directory,

1. ./autoconf2.13
2. mkdir build-scanned
3. cd build-scanned
4. export CC=”ccc-analyzer”
5. export CXX=”ccc-analyzer”
6. scan-build ../configure —enable-debug —disable-optimize
7. scan-build make
8. scan-view /tmp/report …

The CC and CXX are important, because scan-build imposes as a compiler.

Sadly a lot of the issues it found for me seem to be null dereference, that we usually have deliberately. the CRASH method in Yarr, and sometimes it’s a bit too suspicious. But at least the dead store warnings seem to be pretty good.

Here is the full report. I hoped to find some dead function decelerations/definitions, but it looks like it doesn’t do this.

New commit message restrictions

As the result of a discussion on dev.planning, Jesse implemented a new Mercurial hook (same thing as when pushing on a CLOSED TREE)
that disallows certain commit messages. After I made some suggestions in
Bug 506949 he asked if I would like to take over and I happily did. After some waiting, because all the
staff at IT working on Mercurial left, we now have finally deployed the commit hook to mozilla-inbound,
thanks to Benjamin Kero.  It’s not going to actively block commits until Monday 31th October 2011.
The hook just enforces best practices, so you likely won’t notice anything at all. Please read this section “Commit Message Restrictions”.

Here are some perfectly valid commit messages (all from this day):

Henri Sivonen - Bug 696651 part 2 - Establish document.write() insertion position before re-entrant calls can occur. r=Olli.Pettay.
  Includes Bug #####, so good to go.
Ed Morley - Merge mozilla-central and mozilla-inbound
  Starts with “Merge” and is actually a merge.
Paul O’Shannessy - Backout 78c921e2b56b (bug 640136) for causing bug 698162
  Backout followed by 12 hex id.

Important note: If you need to merge a repository after this date, and are hitting this new hook due to invalid commit messages, you can
put “IGNORE BAD COMMIT MESSAGES” into the tip commit message to override all restrictions.

The plan is to deploy this on all repositories that merge to mozilla-central:

• mozilla-central
• integration/mozilla-inbound
• integration/fx-team/
• projects/jaegermonkey
• projects/build-system
• projects/devtools
• projects/places
• services/services-central

(Also first post hopefully synced to planet)

How i got invovled with Mozilla

Just by incident I have been involved with Mozilla for nearly exactly one year now. This is probably a very unlikely story to happen to anyone else.

So in the last year i was focusing on reverse engineering software and one of the skills you obviously are going to need is basic understanding of assembly (x86 in my case). So after a few weeks/months of poking around i got very much interested in compiler optimizations, because that’s something you need to understand when working with optimized binary code. Eventually i came to the point where i wanted to write my own compiler and did some google searching. And one thing that stood out was the work just getting started with Jägermonkey. So i started searching for the code repository (i already knew about Firefox being open source, but i took me quite a bit of time to figure out the meaning and value of Mozilla). I eventually found MXR, but through a bit of luck i then found hg.mozilla.org. But i didn’t stop there at this time Jägermonkey was developed in some user user repository with the name “moo” (seriously!). I followed the progress of this branch about one month till understood how things were done and what kind of interaction between Bugzilla and commits exists. After some more days kangax tweeted this benchmark and i was pretty sure i already knew a way to make it faster. So i sat down created a patch, registered on Bugzilla and created a bug. I got pretty positive responses and joined the #jsapi IRC channel. There i remembered asking for the reason i couldn’t not edit a certain aspect of my bug, about a second later somebody gave my the editbugs permission. At this point this gave me certain feeling of trust and encouragement :)

The story doesn’t get very interesting at this point, i am still involved with work at the JavaScript Engine. After a few months working on the Engine, i got interested in the broader range of things happening in the Mozilla sphere. I started working on MDN and participated at some of the virtual sprints. Luckily enough i was invited to the documentation sprint in Cincinnati, Ohio. I have never been to America before. Because of some family things i also didn’t fly for the last 5 years and never went outside Europe at all. This was a huge surprise and i am still not sure if i understand the amount of luck i had. Just last month i got to enjoy JSconf.eu and meet the people i worked with for nearly one year, but never met. I am still inspired by meeting Brendan Eich, one of my personal “Rockstars” (also i didn’t see him dancing at the party). I also met David Anderson, David Mandelin, Andreas Gal, Gregor Wagner , Jeff Griffiths and many more. One common thing: They were all super nice, friendly and bright.

I also got a computer earlier this year, which i am very thankful for, because i need it for contributing and school.

I do understand that this probably a very unlikely story to happen to somebody else, but i hope somebody might find help in that or a certain kind of encouragement.

At the moment i use a lot of my free-time for Mozilla and i am glad i can. I made the right personal decisions and very lucky about the kind of school i am attending.

;tl,dr I went the hard and long way to get involved, but it was a huge success for me in many aspects of my life.

Also we are always looking for contributors to the JavaScript Engine. If you want the kind of work where you always have a reason the squeeze the last grain of performance, here it is. And if you don’t know that much C/C++, no problem at all. We are happy about JavaScript developers finding certain slow things and what could be optimized. Also bug triage, but this is probably best kept for another blog post.

My first 4 months as mozilla contributer

I made my first contribution to the JavaScript Engine of Mozilla in early October (Bug 601689). After this patch got accepted and committed, my way as contributer began. Nearly all my patches are around the speed/conformance sphere. Really everyone from Mozilla, I had contact with, has been really helpfull and nice. It’s nice to help a project with dedicated and clever people.

I even got an awesome T-Shirt, with a personal greeting card. 

At this point i would like to thank Mozilla (for just being awesome), Hendrik Skupin (for the nice card), the JavaScript Engine Team and Robert Sayrer for offering help in different situations (will specify this in detail someday).