B3 aggregator with hudson

When the Buckminster Aggregator (now B3 Aggregator) came it totally changed how i dealt and worked with our updatesites.

It used to be

• Error prone
• Manual labour
• Searching for updates on updatesites
• No idea if it would suceed when building the updatesite project.

Now that when we have headless aggregator i decided to get rid of this headaches ones and for all. I saw early on that this is a good candidate for integrating into hudson somehow. I figured what i wanted and came to these bullet points:

• Something that monitored different plugin sites
• No more manual searching for updates
• No fixed installations of director (new versions come and go)
• Aggregator headless must be auto updatable, i.e. update itself.
• Build our main updatesite
• Publish the updatesite to our stageserver for testing
• Report differences between different builds of the updatesite i.e. delta.
  
• Run some tests to see that the updatesite actually works.
• Build legacy updatesites as well as new upcoming (we got eclipse 3.4.1/eclipse 3.5.x/myeclipse8.5/Eclipse 3.6)
  
• An interface to the aggregator that helped to achieve above

I implemented an small aggregator helper utility in perl that took some inparameters:

The syntax of this command is:

aggregator_interface.pl [-build | -buildfile] [-buildroot] [-buildtype] [-all]
aggregator_interface.pl [-product | -productpath]
aggregator_interface.pl [-remove]
aggregator_interface.pl [-help | -usage | -version]

Command line options:

-build aggregate an updatesite
-buildfile path to buildfile
-all Aggregate all known updatesites
-buildroot Where you would like the aggregated site to be under [optional]
-buildtype Use VERIFY | BUILD | CLEAN | CLEAN_BUILD to what kind of build you need [optional]
-product Name the eclipse product you would like to install ex: "org.eclipse.sdk.ide"
-productpath path where eclipse product will be installed
-feature Name the feature you need installed

-remove will remove (rmdir) folder you say
-help Shows help text.
-usage Shows how to use aggregator_interface.pl

-version Shows version information

Integrating this into Hudson works very well. The perl script handles the installations of director and the headless aggregator. Based upon what you specified it goes out an does what you want.

I have 1 * .b3aggr file / updatesite. All *.b3aggr files is stored in our VCS system (clearcase) the perl script with the -all option reads all *.b3aggr files and runs out and fetches these to an internal fileshare.

We then have one *.b3aggr file that consumes all these internal fileshares that was just aggregated. It uses custom categories etc for our internal needs. This means we have no network problems as building this is all internal.

When the update site is done aggregating, Hudson runs an an ant file thatcreates some html documents then uploads this to our internal eclipse update stage area via winscp.

Scripting with winscp is easy

example:

# Automatically answer all prompts negatively not to stall
# the script on errors
option batch on
# Disable overwrite confirmations that conflict with the previous
option confirm off
# Connect using a password
# open user:password@example.com
# Connect
open userid:password@server/opt/apache-2.0.59/htdocs/staging/products
#remove /products/eclipse3.6
rmdir eclipse3.6
mkdir eclipse3.6
cd /opt/apache-2.0.59/htdocs/staging/products/eclipse3.6
#update with newly built galileo
synchronize remote %buildpath%\site.p2 -mirror
exit

Now Hudson has published the new updatesite on our staging machine for testing

Eclipse and proxy support

Proxys and Eclipse, i think this issue has caused me more headache then any other issue especially in support. In corporate installation of Eclipse you always have an proxy server, nobody is allowed to go directly to the internet.

Corporate/Goverment development shops most likely have security staff who has a policy telling you whats allowed and whats not allowed. Letting the users download their own eclipse plugins is usually a big no no. Both from a security standpoint but also from a configuration standpoint.

So i have set my task of locking down the proxy to only allow direct connections i.e. connections internally and making sure the user cannot bypass this in any easy way. Setting this in plugin_customization.ini does not work, tried:

org.eclipse.core.net/proxiesEnabled=false
org.eclipse.core.net/systemProxiesEnabled=false
org.eclipse.core.net/nonProxiedHosts=*.host.com|localhost|12 7.0.0.1

did not work. asking for help:

http://www.eclipse.org/forums/index.php?t=msg&th=169365&

resulted in that i needed to "patch" two plugins.

After consulting my friend google i found the ui preferences in

org.eclipse.ui.net / ProxyPreferencePage.java

lock down the ui combo box with:

providerCombo.setEnabled(false);

and the default active provider in

org.eclipse.core.net / proxySelector.java

Default for us Eclipse returns the NATIVE_PROVIDER which in turn works , i.e. Eclipse somehow finds our proxy settings and lets you download plugins from the internet. Not what we want!

so i changed the "return NATIVE_PROVIDER;" to "return DIRECT_PROVIDER;"

These settings now returns the direct provider as default and lock the ui combobox so you can't change them.

This will be built into my Eclipse product based upon Helios.