Loading a page containing Javascript in a Chrome extension without displaying the page

Question

I'm building a Chrome extension that needs to see the results of a Google Groups topic search (the page URL is https://groups.google.com/forum/?fromgroups=#!searchin/opencomments-site-discussions/subject:{some topic} and I can tell if there is or is not a match by looking for a certain classname). Unfortunately, the field it's looking for doesn't get created until after the page's Javascript has run, so I can't simply look at the raw HTML from the page via

try {
  var request = new XMLHttpRequest();
  request.open("GET", url, false);
  request.send(null);
} catch (e) {
  ....
}
if (request.status == 200) {
  var rawText = request.responseText;
  ...
}

Does anyone know if it's possible to load the entire page, including executing the Javascript, without displaying the page? Been pulling my hair out on this problem.

Answer 1

Have a look at the offscreenTabs API (currently experimental).
After creating a tab using chrome.experimental.offscreenTabs.create, you can use chrome.tabs.sendMessage or chrome.tabs.executeScript to do whatever you want.

If you need inspiration, check this full demo.

Per specification, an Offscreen tab can only be created within a real tab (otherwise, you'll get "Error during experimental.offscreenTabs.create: No current tab found"). In my other demo, I used an options page for this purpose. If you don't want to use such a page, create a new tab using chrome.tabs.create, do whatever you want, then use window.close() to close the tab.

So, you're going to open a (hidden) tab after all (though the page itself is not visible). With the current APIs, there is no other way to achieve rendering a hidden tab without opening a tab (unless you want to render the page in an iframe at background page...). If you don't mind the actual tab from showing up in the tabs bar, use chrome.tabs.create({active:false, url:'...'}). If it's important to keep the tab truly invisible to the user, proceed with the following demo:

Demo usng the offscreenTabs API

In this example, I'm going to show an alert which contains the invisible tab's title. I can show this alert from the content script right away, but instead, I'm going to pass the data to the background page, to demonstrate the feature:

Background script (background.js):

 chrome.extension.onMessage.addListener(function(message, sender, sendResponse) {
     var tab = sender.tab;
     if (tab && tab.index === -1) { // <-- Detect offscreen tab
         if (message.method == 'title') {
             // Close helper tab:
             chrome.extension.sendMessage({method: 'finishedTab', id: tab.id});
             alert('Got title from ' + tab.url + ' :\n' + message.result);
         }
     }
 });

 chrome.browserAction.onClicked.addListener(function() {
     chrome.tabs.create({
        active: false,
        url: chrome.extension.getURL('launcher.html')
     });
 });

Content script (contentscript.js):

chrome.extension.sendMessage({
    method: 'title',
    result: document.title
});

launcher.html contains <script src="launcher.js"></script>. launcher.js:

var offscreenTabId;
// Close the tab when the stuff is done
chrome.extension.onMessage.addListener(function(message) {
    if (message.method == 'finishedTab' && message.id === offscreenTabId) {
        window.close();
    }
});
// Create invisible tab
var createProperties = {url: "https://stackoverflow.com/"};
chrome.experimental.offscreenTabs.create(createProperties, function(offscreenTab) {
    offscreenTabId = offscreenTab.id;
});

Manifest file:

{
    "name": "Get information from DOM in an invisible tab",
    "description": "See https://stackoverflow.com/q/13714002",
    "version": "1",
    "manifest_version": 2,
    "permissions": ["experimental", "<all_urls>"],
    "background": {"scripts": ["background.js"] },
    "content_scripts": [{
        "js": ["contentscript.js"],
        "matches": ["<all_urls>"]
    }],
    "browser_action": {}
}