Implement privileged DOM/XUL access if possible (was: "Error when accessing datasources")

[ghost]

Hello Brett,

I received your email about my precedent issue and again thank you very much for helping.
I tried your solution but didn't manage to have it working. I get 2 errors : one is quite much a warning telling that components are deprecated, will be soon removed and another that prevent my code from working.

Here is my code :

Datasource.build = function(id, url, params, callback) {
    AsYouWish.requestPrivs(['chrome'], function (chrome) {
        var elem = $(id);
        var uri = new Datasource(url, params).render();
        var rdf = chrome.Cc['@mozilla.org/rdf/rdf-service;1'].getService(Components.interfaces.nsIRDFService);
        var datasource = rdf.GetDataSource(uri);

        Datasource.createManagers(id, elem);
        Datasource.registerCallback(id, callback);

        Datasource.clear(id);

        datasource.QueryInterface(Components.interfaces.nsIRDFRemoteDataSource);
        datasource.QueryInterface(Components.interfaces.nsIRDFXMLSink);

        DatasourceManager.push();

        elem.database.AddDataSource(datasource);

        datasource.addXMLSinkObserver(Datasource.observers[id]);
    });
}

The warning is triggered by
var rdf = chrome.Cc['@mozilla.org/rdf/rdf-service;1'].getService(Components.interfaces.nsIRDFService);
and the real error is triggered by
elem.database.AddDataSource(datasource); telling me the code isn't allowed to wrap ...

Do you have an idea about this ?

Thanks again,
Kind regards
Jérémy

[brettz9]

Try this instead:

// "components" must be lower case
var rdf = chrome.Cc['@mozilla.org/rdf/rdf-service;1'].getService(chrome.components.interfaces.nsIRDFService);

or this:

var rdf = chrome.Cc['@mozilla.org/rdf/rdf-service;1'].getService(chrome.Ci.nsIRDFService);

or better yet:

var Cc = chrome.Cc, Ci = chrome.Ci;

var rdf = Cc['@mozilla.org/rdf/rdf-service;1'].getService(Ci.nsIRDFService);

// You can also replace Components.classes or Components.interfaces elsewhere in 
// your code with Cc and Ci, if defined as above

The other error may be more serious as my code has tried to wrap all objects to avoid security complaints from Firefox that may prevent them from being usable in the future while still allowing you to have full access. That being said, it is possible after some improvements I hope to look at this weekend, that the wrapping could be more complete (though I'm not sure). Also, is your code above complete so I will have everything I need to test it? What is it trying to do exactly--i.e., how will I know it is working? Also, do you have a documentation page for the RDF API at hand, so I can see what you are trying to do--my head always hurt trying to use RDF. Are you sure you really need it?

[brettz9]

Actually, for all cases of Components.interfaces, I would suggest using Ci as defined in the code just sent because I am also not sure whether the Components.interfaces still available within HTML is of the same kind exposed to the Addons SDK which my addon is making available.

[brettz9]

And the exact error message might be helpful too...

[ghost]

Hello again,

Thank you for you so fast answer ! I tried what you said but still get the same error.
My code is a class that aim to manage rdf datasources in for xul container elements.

Here is the full class code :

var DatasourceManager = function() {
    var _count = (_count == undefined) ? 0 : _count;
    var _pushing = null;
    var _poping = null;
    var _erroring = null;

    return {
        getCount: function() {
            return _count;
        },
        push: function() {
            ++_count;

            if (_pushing != null) {
                _pushing(_count);
            }    
        },
        pushing: function(callback) {
            _pushing = callback;
        },
        pop: function() {
            --_count;

            if (_poping != null) {
                _poping(_count);
            }    
        },
        poping: function(callback) {
            _poping = callback;
        },
        error: function(element) {
            if (_erroring != null) {
                _erroring(element);
            }    
        },
        erroring: function(callback) {
            _erroring = callback;
        }
    };
} ();

function Datasource(url, args, path) {
    var _root = 'http://' + window.location.hostname + '/';
    var _url = url;
    var _path = path || 'php/';
    var _args = args || {};
    var _value = '';

    function parseArg(key, value) {
        _value += (_value == '') ? '?' : '&';
        _value += encodeURIComponent(key)+'='+encodeURIComponent(value);
    }

    function parseArgs() {
        var key;

        _args.ztimer = String(Math.random()).substr(0, 10);
        _value = '';

        for (key in _args) {
            parseArg (key, _args[key]);
        }    
    }

    this.setArg = function(key, value) {
        _args[key] = value;
    }

    this.setArgs = function(args) {
        var key;

        for (key in args) {
            this.setArg(key, args[key]);
        }    
    }

    this.render = function() {
        parseArgs();

        return _root+_path+_url+_value;
    }
}

Datasource.observers = new Object();

Datasource.build = function(id, url, params, callback) {
    AsYouWish.requestPrivs(['chrome'], function (chrome) {
        var Cc = chrome.Cc, Ci = chrome.Ci;
        var elem = $(id);
        var uri = new Datasource(url, params).render();
        var rdf = Cc['@mozilla.org/rdf/rdf-service;1'].getService(Ci.nsIRDFService);
        var datasource = rdf.GetDataSource(uri);

        Datasource.createManagers(id, elem);
        Datasource.registerCallback(id, callback);

        Datasource.clear(id);

        datasource.QueryInterface(Ci.nsIRDFRemoteDataSource);
        datasource.QueryInterface(Ci.nsIRDFXMLSink);

        DatasourceManager.push();

        elem.database.AddDataSource(datasource);

        datasource.addXMLSinkObserver(Datasource.observers[id]);
    });
}
Datasource.refresh = function(id, callback) {
    DatasourceManager.push();

    Datasource.registerCallback(id, callback);

    $(id).builder.refresh();
}
Datasource.clear = function(id) {
    var elem = $(id);

    elem.datasources = 'rdf:null';
    elem.datasources = '';

    var sources = elem.database.GetDataSources();

    while (sources.hasMoreElements()) {
        elem.database.RemoveDataSource(sources.getNext());
    }
}
Datasource.read = function(id, about, field, defaultValue) {
    AsYouWish.requestPrivs(['chrome'], function (chrome) {
        var Cc = chrome.Cc, Ci = chrome.Ci;
        var rdf = Cc['@mozilla.org/rdf/rdf-service;1'].getService(Ci.nsIRDFService);
        var target = $(id).database.GetTarget(rdf.GetResource(about), rdf.GetResource(field), true);

        return (target instanceof Ci.nsIRDFLiteral) ? target.Value : defaultValue;
    });
}
Datasource.createManagers = function(id, elem) {
    Datasource.observers[id] = {
        callback: null,
        element: elem,
        onBeginLoad: function(sink) {},
        onInterrupt: function(sink) {},
        onResume: function(sink) {},
        onError: function(sink, status, msg) {
            DatasourceManager.error(this.element);
        },
        onEndLoad: function(sink) {
            DatasourceManager.pop();

            this.element.builder.rebuild();

            if (this.callback) {
                this.callback(this.element);
                this.callback = null;
            }
        }
    };
}
Datasource.registerCallback = function(id, callback) {
    if (callback != undefined) {
        Datasource.observers[id].callback = callback;
    }    
}

You can use it like that :

Datasource.build('logSearchs', 'sources-log.php', {type: 'search'});
// arguments : xul element id, source url, source query parameters

I am encountering this error :
On line 122
Error: Permission denied for http://admin.cartes.mtgfrance.dev to create wrapper for object of class UnnamedClass
[Break On This Error]

var sources = elem.database.GetDataSources();

Am I doing something wrong ?

Since I'm very glad of you to help me and this is for professional purpose I'll be happy to reward you at the end !
Any paypal account for a donation ? :)

[brettz9]

Hi Jérémy,

(Sorry for the delayed reply--had some rest to catch up on this weekend, and just have some free time now.)

I hadn't looked carefully enough at the code earlier. I see you are trying to access a XUL tree element, and AsYouWish does not allow arbitrary use of XUL elements within your HTML, as the addon is using, and is itself, part of the Addons SDK, which is really meant to avoid the need for XUL (although there is no clear direct substitute for RDF-driven trees, I have asked the developer of some tools which had supported XUL emulation in HTML: ilinsky/xbl#9 about the link to this information in case it might help; if not, maybe you could move away from RDF which is no longer fashionable and move to something like this: https://github.com/ilinsky/xbl/blob/master/examples/tree/tree.html (live example at http://xbl.googlecode.com/svn/trunk/examples/tree/tree.html ) or any of the other tree interfaces out there, like for jQuery.).

If not, you could see https://developer.mozilla.org/en-US/docs/Using_Remote_XUL about whitelisting your XUL page. I haven't worked with Remote XUL to know whether:

1. It is still possible to whitelist it (the continued presence in the docs and on https://addons.mozilla.org/en-US/firefox/addon/remote-xul-manager/ seems to suggest it is), and
2. I don't know whether this would be sufficient to avoid need for AsYouWish altogether, and if not, whether you could put the XUL tree in a separate file and whitelist that file (from AsYouWish even) per the Remote XUL docs, and then let AsYouWish populate it OR whether there is some other way to wrap up XUL elements from the SDK APIs for placement within otherwise unprivileged HTML.

Since it appears that this is not really a bug of AsYouWish, I'm closing the issue, though you can feel free to continue commenting here. You could email me at [email protected] if you need paid support help to get this working for you, but this is really out of scope for me for improving the addon itself.

I've added a FAQ entry regarding this question as well: https://github.com/brettz9/asyouwish/wiki/Developer-FAQ#wiki-xul

[brettz9]

If you do try to work on this on your own, please feel free to share the results of your investigations here so I can share this with others on the wiki (which is locked for security reasons).

[brettz9]

I just now tested the whitelisting of a separate XUL page, and it worked:

<iframe id="remoteXUL" src="remoteXUL.xul"></iframe>
<script>
AsYouWish.requestPrivs(['chrome'], function (chrome) {
    var Cc = chrome.Cc, Ci = chrome.Ci;
    Cc['@mozilla.org/permissionmanager;1'].getService(Ci.nsIPermissionManager).add(
        Cc["@mozilla.org/network/io-service;1"].getService(Ci.nsIIOService).newURI(
            'http://127.0.0.1/tests/remoteXUL.xul', null, null
        ), 
        'allowXULXBL', Ci.nsIPermissionManager.ALLOW_ACTION
    );
    var remoteXUL = document.getElementById('remoteXUL');
    remoteXUL.onload = function () {
        alert(remoteXUL.contentDocument.getElementById('test').value); // abc
    };
});
</script>

for this simple XUL test file (placed at http://127.0.0.1/tests/remoteXUL.xul per the above code):

<window xmlns="http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul">
<textbox id="test" value="abc" />
</window>

...so maybe you could try the approach of including the XUL tree inside an iframe.

[brettz9]

I've updated the issue (visible on Github) showing that one can use iframes to interact with the remote XUL (at least same domain, so maybe this could work for you).

[ghost]

Hello,

I'm using the Remote Xul Manager since many months but I doesn't solve the problem for the removal of enablePrivileges.
Actually I'm working on a full remote XUL project, it's not XUL embedded into some HTML.

Thanks you very much for all your efforts, that's too bad I can't solve my problem with your (however so great) addon but I think I won't have another choice than to stay with FF 16 for this project !

I could consider to include my project into a HTML iframe but since it's a very big application I think many things will then broke and I think this solution is probably even worst than to remain with FF 16.

Then, again congratulations for your addon and thanks again for what you did for me !

Best regards,
Jérémy

[brettz9]

Very sorry to hear that Jérémy. I know it must be disappointing to find that out after more attempts with it. I was going to try to offer some hope for the future if XBL were to become standardized as earlier was expected (and perhaps people adapting bindings for all of XUL), but I see now from http://www.w3.org/TR/xbl/ that browsers have no plans to implement this standard (and https://github.com/ilinsky/xbl doesn't offer them). It seems increasingly clear to me that JavaScript is the only language worth using for long-term apps (as far as one can predict into the future anyways)--including over static HTML (and perhaps even CSS) itself, given the perpetual limits on HTML modularity (and for the inherent advantages of a shared syntax). Please do feel free to keep me posted if you do revisit this or attempt to use AsYouWish for any new development work.

[brettz9]

I am thinking of a new approach which might possibly work (mentioned also at http://stackoverflow.com/questions/21446737/granting-website-privileges-to-access-cross-domain-iframe-content ). I might be able to allow an HTML page to request privs remotely (since I understand remote XUL no longer works) and then refresh inside my OWN chrome:// URL which will give the HTML iframe privileges, and presumably extend these privileges to any XUL children (including nested XUL, I would hope). Just an idea now, but letting you know... For now, I'm reopening the issue (and renamed it).

[xthdraft]

Hi Brett,
Your asyouwish extension works flawlessly for my purposes (a desktop word processor in html & javascript) and I'd like to thank you again for your super work. However, as I'm planning to work on my program further in the near future I wanted to bring up one concern.

As we all know, Mozilla has a habit of changing its code very frequently (which is in the main a good thing - firefox is a great product, I'm not complaining). But these changes can be very damaging to app developers. The classic case for me was the disabling of enablePrivilege from v17 or thereabouts. Fortunately, your extension solved this "deficiency" (which Mozilla of course would call a "security enhancement").

Now, here is my concern. Is it possible that Mozilla might one day change its code base in such a way as to make your great extension unworkable? Basically, we're still at mozilla's mercy, yeah? Not to mention the other browsers out there who might all have their different caveats on local file i/o.

Therefore, would it not be wise to have another type of strategy to enable file i/o on the desktop?

My idea, which I already use to some extent using apache/localhost, is to use python in a local-server context. (Since my app already necessitates the user having python installed for some other functionality.)

I'm thinking of simplehttpserver in the same folder as the html app. So, for example, you could have readfile() and writefile() functions using an ajax call to python's little teeny server which would use tkinter's filepicker for the navigation, and then python handles the 2-way transfer of the contents with the html. That sort of thing.

This would obviously not be as seemless or elegant as asyouwish, but my slight concern is that your solution might be vulnerable to Mozilla's possible changes down the track. Thus the need, maybe, for an insurance policy.

What I'm basically asking you therefore is whether you think asyouwish is a bullet-proof long-term answer to firefox's enablePrivilege problem.

Best regards,
Eamon Byrne

[brettz9]

No, unfortunately, I can't guarantee the solution will continue to work, whether partially or at all. Even now it appears that XHR may break in the Nightly versions of Firefox--and Mozilla doesn't seem like it is going to make any efforts to respond to any of my reports about breakage (I haven't even gotten an add-on review for AYW from them after more than well over a year of waiting!).

The one room for hope is that, if there is breakage, by using the SDK--to the extent Mozilla indicates its APIs as being stable and sticks to that--you should be able to convert AsYouWish code without too much extra trouble into a Firefox add-on.

However, if you want to support other browsers, the Mozilla SDK API is not mentioned as being on its way to being standardized or anything (although Web APIs are progressively taking on more capabilities, albeit sometimes not reaching the desktop, at least right away). I did make a request for this at https://bugzilla.mozilla.org/show_bug.cgi?id=848647 .

While AsYouWish, in using the SDK, lets you use browser-specific APIs (e.g., obtaining a list of the user's opened tabs) which are not available to server-based solutions, it seems in your case, for file access, you will not need them. And unlike AsYouWish or a Firefox add-on, these would work cross-browser, so a server solution might be in your interest.

While a Python solution might end up working for you, I'm personally a fan of server-side JavaScript (as in Node.js) so that one can share code between the client and server, utilize the same development tools, etc. Besides that, there are Node.js-based solutions which allow you to directly use server-side APIs without needing to set up some kind of Ajax communications:

• https://github.com/rogerwang/node-webkit
• http://appjs.com/
• http://deskshell.org/

(I haven't used these myself, but I know at least the first one is pretty popular.)

Take care,
Brett

[brettz9]

You may also be interested in https://hacks.mozilla.org/2012/07/why-no-filesystem-api-in-firefox/ (this is also interesting too, but I don't think file system support exists yet, so this couldn't be used either: https://hacks.mozilla.org/2013/10/progress-report-on-cross-platform-open-web-apps/ )

[xthdraft]

Thanks Brett. The possibility of Mozilla shifting the goalposts is a worry. I've been all day looking at tkinter as a possible emergency back-end for file i/o, but have decided it's too clunky, since tkinter is not really designed for embedding in a html page.

This leaves node.js and indexedDB as a couple of other candidates as alternatives to ASYOUWISH - which is a bummer since AYW was exactly the perfect solution for what should really be a very simple problem.

I really can't understand why browsers don't have pure desktop versions stripped of the connectivity to the outside world - using the full capabilities of the various languages. But that's the whole issue isn't it, javascript is so great except you can't do a, b and c. And python is also great except you need x, y and z. Not to mention all the other competing cogs and wheels out there. And so you have all these zillions of wheels being constantly reinvented, and they're all really great except you have to put 4 completely different wheels on your car to go anywhere. So the car as we know it today is almost still completely f*ckd. The world's programmers are getting around on ricketty bicycles Brett. Any worse and it gets serious, I kid you not.

Ok, back to more creative ranting of a personal nature. See ya.

[brettz9]

I think you are absolutely right. And then you get people telling you, "Use the right tool for the right job", which, while perhaps true when you have no choice, is completely ignorant of the fact that there is nothing so special about any of these particular tools which does not allow them to be adapted with the right effort. And unlike the typical "tool" which is both incapable of being molded to your needs and without cost to learn to use, programming languages are more like human languages.

With human languages the situation is currently the same as well, imo--any could serve as an official world auxiliary language, but people focus all their time on studying different ones and priding themselves on this rather than working to get ourselves toward a global meeting that would decide on a standard that would work everywhere.