Montreal demo camp next week

The Montreal demo camp will be on next Tuesday, Nov 29th! Don't forget to register and propose a session as there are still many spots available.

See you there.

Eclipse, fashion boutique

Beyond the awesome OSS software it provides, Eclipse is the place to get dressed! As an evidence, here is my personal collection of Eclipse T-shirts gathered over the years of conferences, camps, Lynn bribing, etc.

Missing from this list: the long sleeves jeans shirts from eclipsecon 2007(?), the very first Eclipse hat from OTI/IBM, some GSoc shirts, and the EclipseCon exercise shirts, but I never got those because I'm usually too "tired" for the early runs.

With each of them come some good memories and I still wear them proudly :)

What I believe is the very first Eclipse shirt

p2 and equinox "office hours"

You have beginner or experienced-level questions about equinox or p2? You want to contribute to equinox or p2? Just want to chat with us?
Join us on our weekly office hours every Tuesday at 1pm EST on the #equinox-dev channel on IRC (http://wiki.eclipse.org/IRC).

During these meetings equinox and p2 committers, contributors and aficionados will answer questions on any topic you come up with. Questions to be answered on a first come, first served basis.

See you there.

Ps: This is inspired from the office hours held by the Jenkins team

More on p2 and OBR

When I set out to write my previous blog post about why not OBR, the intent was to give, with a catchy title, an historical perspective on p2 (thus the recent title update) but also to answer people who have been asking for years this question

(and yesterday night another one of those came in).

The point was not to do a comparison of p2 and OBR as they are today, and especially not to claim that OBR is not good today. Both technologies have pros and cons. My statements about OBR were a reflection of the situation as it was ~5 years ago. To be honest, I have not been following the recent development on the spec, just heard some rumours.

When I wrote the previous post, I also wrote two additional paragraphs that I had not published. One was about our absence of involvement in the spec process and one about going forward. So here they are:

Could we have collaborated on the spec and implementation?

Maybe, but at the time the interest around OBR was rather low at the OSGi alliance, we did not had the bandwidth to drive improvements to the specification since we were focusing on shipping running software (the first release may not have been the best thing, I agree), and my management at the time was not up for the work, distraction, and frustration it could have represented. It needs to be remembered that some of us had been involved in the OSGi spec process to drive changes (adding fragments, bsn, require bundle, etc. Yes all these mostly were driven by the equinox team) and it had been a painful experience that left the people involved with scars, and made some of them the ongoing target of criticisms because of the changes they pushed for.

A couple years later (~2009?), OBR got brought back the forefront in the context of the OSGi Enterprise Expert Group. At this point, I have been involved in the spec work even to a point where I was the lead. Because of my lack of time, and the problems scoping the work, my contributions have been limited and then I stopped being involved as Sonatype withdrew from the OSGi alliance.

Is there a way forward?

Until a publicly visible draft of OBR is available, I can't provide a definitive answer. However implementing OBR on top of p2 has definitely been discussed several times and is not out of question especially if the specification stays focused on providing a resolver API and a repository API.

Now time, energy and contribution will tell...

PS: there are things that need to be corrected from Neil's post, but my other commitments are calling so it will have to wait.

Why not OBR? - An historical perspective on p2.

Many times I've been asked why p2 got created when there were other solutions around, namely OBR. To understand why p2 got created, it takes a little bit of understanding on the requirements that drove the work on p2, but also some history.

So let's start with the requirements.

The work on p2 started in 2006 / 2007 at at time where companies contributing to Eclipse were wondering how to increase their ROI. For this, they were all sure that having an easy path from the free Eclipse distress to their cool add-ons or full suite of products would be the solution. The term coined for that was "on-ramp". Nowadays what best embodies this is the Eclipse Marketplace and its client.

It is in this context that IBM spearheaded an internal effort to replace the aging update manager. The requirements were to provide a "provisioning platform" that would allow to install the complete Eclipse SDK from scratch (not only bundles and features like the old solution was), and also deal with more native things like dlls, .net assemblies, rpms, etc. and provide desktop integrations. In addition to that, the solution was expected to:

- allow both fine grained provisioning (just want to install a bundle) or large grained provisioning (install a complete product) -- this is why we used the term "right-grained provisioning" to describe a solution that would let its user decide the right granularity for himself.

- provide a "transactional" update, meaning that a failed or cancelled install could not leave the system in an unknown state

- allow to install and uninstall without having the provisioned system be up and running

- separate the metadata from the payload

- support mirroring

- …

With those requirements, I spent a fair amount of time looking at what was being provided in the Java and OSGi ecosystems, as well as what could be offered from the linux package managers and other esoteric technologies. This was a phase of research were I was harvesting as many ideas as possible, comparing them with each others, matching them with my requirements. It is really this phase that defined much of what p2 has become. Without being exhaustive, here are some of the things that I looked at: SDD (http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=sdd) from oasis, OBR from OSGi, Deployment Admin from OSGi, JSR 124 (http://jcp.org/en/jsr/detail?id=124), another similar JSR for embedded device (can't remember the number), Maven, debian, conary (http://wiki.rpath.com/wiki/Conary), RPM, numerous Java installers, etc. From their I went down the path of research papers for dependency resolvers, looked at constraints programming, and was lucky to come across the paper from Tucker that studied the usage of a SAT solver to resolve install problems (unfortunately his work never made it to production).

So why not OBR?

OBR was not up to the task. OBR was a repository format, a resolver, and a downloader but each of these element was far away from being sufficient to build what we needed on top.

- OBR focus was bundle centric. No "group" (aka features), no native files, …

- The repository format was fixed and was XML only, no possibility of extension.

- Each bundle listed in the repository was referring to the payload (the actual jar for the bundle) using a fix URL rather than abstracting the artifact location in a coordinate -- like Maven does with its concept of GAV or like p2 does with "artifact keys" -- which makes mirroring just plain painful (I know it is possible to provide URL handlers, but come on…).

- The provided API was not allowing for transactional state change (some bundles could be installed, but not all the required ones if the download failed)

- The metadata did not accommodate the expression of what needed to happen to the bundle (e.g. should it be started, set the start level, etc.) and thus even less to other things.

- The resolver always insisted on installing all the fragments available (which in complex repos like Indigo means that installing JDT will bring in parts of WindowBuilder since it has some fragment to JDT UI iirc)

- …

As you've seen, a fair amount due diligence to OBR and plenty of technologies had been done. Rather than being a thing that just came to be by itself, p2 claims lineage from pretty much all these technologies, and the whys could be repeated and inverted at will :).

PS: Short lineage: InstallableUnit and Touchpoint terminology comes from SDD, parts of the transactional aspect of the p2 engine is inspired from OSGi deployment admin, the list of phase is from debian, the metadata content is inspired from OBR and OSGi Manifest, Maven for the idea of coordinate system...

Update: Changing the title to reflect the intent of the post

Tame your Java dependencies

As an application becomes more modular but also bigger, one often ask him/herself questions like:

- is there a more recent version (or a released version) of this library ?

- is this library consistently used across the project?

- why is this library being used?

To address these, Milos Kleint, Matthew Piggott and I have been working on an Eclipse plug-in that allows to answer those questions directly from within the IDE and for any Java project (JDT, Maven, PDE, …). Even though the plug-in will only be made available after Thrusday's webinar, here is a introduction to it showing how the questions are being answered.

Is there a more recent version of this library?

This information is presented in the "Dependency Hub" view. This view is divided into two sections, an overview on the left, and a details section on the right. The overview presents a list of all the libraries used by the projects contained in the workspace and colour coded arrows representing the various updates available, whereas the details provides a list of the available versions.

Is this library consistently used across the project?

This information can quickly be found in the overview section of the view.

Why is this library being used?

When you find a suspicious dependency or some inconsistency, the search facility will identify how the library is brought in and present the results in the Eclipse search dialog. It not only searches in the direct dependencies but also in transitive dependencies when used with a Maven project.

Finally, another cool feature we've added but is Maven specific, is the ability to modify the pom.xml to update to a new library.

To conclude, we are pretty pleased with the tool and hope you will enjoy it to. This first version is beta and free, and over the next few months we will be rolling out more updates with new features.

Oh my god, but where is m2e?

Since yesterday's release of Indigo, there seems to have been some confusion about where m2e can be found. So here are some answers for you:

- m2e is now a project of the Eclipse Foundation: http://www.eclipse.org

- The landing page for the project is: http://www.eclipse.org/m2e

- Bug reports go there: https://bugs.eclipse.org/bugs/enter_bug.cgi?product=m2e and

- User mailing list: https://dev.eclipse.org/mailman/listinfo/m2e-users

- Dev mailing list: https://dev.eclipse.org/mailman/listinfo/m2e-dev

Where can you download m2e from?

- m2e comes prepackaged in the Eclipse IDE for Java Developers

- it is also available from the indigo repository (http://download.eclipse.org/releases/indigo) and can be installed using Help > Installed New Software...

Why does m2e not ship in the Eclipse IDE for Java EE Developers package?

Long story short, this is because at the time we made the decision where to include m2e, the future of the m2e/WTP integration was uncertain. However, since then the situation has changed with Fred Bricon being hired by JBoss and have him be focused on this piece of technology.

In the meantime, I have opened an enhancement request https://bugs.eclipse.org/bugs/show_bug.cgi?id=350150 to get m2e added to the JEE package.

So in short, m2e is here, it is available, go use it.

From Helios to Indigo is easy

Indigo is now out. You've been running Helios for quite a while and have added tons of plug-ins on top of it and you don't really want to have to craft that install again. Here are below a few possibilities to ease with the update:

Possibility 1 - Morphing Helios into Indigo

The idea here is that you will upgrade your Helios install into an Indigo one. If all plug-ins have been playing nicely in specifying their version ranges and using APIs, this should just work. Otherwise you will get presented with errors messages having unsatisfiable dependencies. Anyway, here are the steps:

Step 1: Add the http://download.eclipse.org/releases/indigo to the set of repositories (Preferences > Install Updates > Available Software Sites).

Step 2: Search for updates: Help > Check for Updates. Note that this will search for updates for all the plugins that are installed. If you want a more selective update, try Help > About > Installation Details and select the entries you are interested in updating among the elements listed in the Installed Software tab (see pictures below).

Step 3 : Once prompted with the available updates, select the desired items, click next a couple times and patiently wait for the bytes to download to your machine. Then restart and enjoy!

Possibility 2 - Start fresh and install plugins from an old install

The idea here is that you want to keep your old install intact, download a new indigo package (http://download.eclipse.org/) and just adds the existing plugins to your install. With Indigo this is now made super easy!

Step 1: Download and unzip your Indigo install: http://download.eclipse.org

Step 2: Import your plug-ins from your previous install by doing File > Import > Install > From existing install, pick your old install, select the plug-ins and there you go! You can chose to go piece meal, or chose to pick several entries, in any event dependency analysis will be performed to check the validity of the install.

Step 3: Click next a couple times and patiently wait for the bytes to download to your machine. Then restart and enjoy!

Montreal democamp notes

Eclipse democamp in Montreal came and went. It was definitely a great event with a lot of participants and great presentations. Here is a quick recap of what was there.

Remote provisioning with JMS

In this demo, Anthony demo'ed how he used the p2 API to remotely manage with JMS multiple nodes running p2. To help with the visual aspect of the demo, the managed application was an RCP app show casing the dynamic addition and removal of a menu, but the application scheduled to go in production in a couple months will be handling regular OSGi apps.

After having heard about this project on the p2 ml, I was happy to see it live!

z/OS synchronization

Here, Johnny demonstrated how the Eclipse Team API can not only used to work against SCMs but can be used to keep a remote system and a local system in sync.

In this case, he was demonstrating how to keep a z/OS system in sync with resources in the workspace reusing concepts like commit, update, sync, etc.

The presenter was pretty happy to note that he has been able to implement everything he needed *only* using the API and that the API was well designed and well documented.

PDE feature dependencies visualization

This time Johnny demonstrated the work of one of his coop that consists in adding to the PDE visualization project the ability to view inter feature dependencies.

In this case, they started by implementing a clean Model / UI separation where the model is agnostic of features and simply uses a generic graph. With this graph based model, they have been able to implement well known graph algorithms such as shortest path, or cycle detection.

This tool will definitely be a nice addition to the PDE visualization and I'm already dreaming of a p2 extension for it :)

Eclipse@CSA

During this presentation Regent from the Canadian Space Agency demonstrated how the various Eclipse frameworks (RCP, EMF, etc.) helped the agency deliver in no time a large set of products ranging from a way to operate highly autonomous bots to space operations planning software. Regent was strongly recommending the usage of EMF as a time saver.

The good news is that some of the components the agency has developed will also be contributed back in the open source, but this is a long process since this is a first time.

R4e - Review for Eclipse

Alvaro and Sebastien demonstrated review for Eclipse, a nicely integrated plugin that let you do code review directly from within Eclipse.

I had not realized but code review in Eclipse seems to be a hot topic, so hot that there are currently 3 projects at eclipse to deal with the topic: Mylyn reviews, Gerrit and review 4 eclipse…

Debugging in CDT

For years, C/C++ developers had been asking for CDT to catch up with JDT. A couple years ago Doug Schaefer was demonstrating CDT had finally caught up with the possibility to do refactoring, better launching, etc. What I saw yesterday simply blew JDT debugging support out the water! Marc showed some of the new and noteworthy from CDT 8:

- debugging an app that is *not* a project in the IDE

- attach to a running process

- smart way to compare data executed between multiple threads (second variable view, expression view, etc.)

- improved support for multicore (show the core)

Last point, these guys are working hand in hand with the GDB team to continuously push the boundaries and more multicore enhancements are in the pipe. Again kudos :)

Sharing preferences

Domenic demonstrated a new plugin developed at Ericsson to share Eclipse preferences across a team. The solution is quite simple and efficient: one or multiple preference files are shared on a server, the plugin gets them and apply them. The plugin also offers the ability to diff the preferences currently active from those that were in the reference file.

The code is currently available in bug https://bugs.eclipse.org/334016 and discussions will happen in 3.8 to see where it best fit.

Ericsson's p2 installer

Domenic demonstrated a customized version of p2 that dealt with sharing bundles across all the installs of Eclipse for all their users. The use cases for that stemmed from both a disk space saving and a control perspective. Since all the tools are installed on a shared mounted drive it is beneficial for Ericsson to have a "company-wide bundle pool" such every user of Eclipse does not end up with its own copy of Eclipse. As for control, this bundle pool is also used as a catalog from which the user can pick what he wants. Still the tool let the user install anything but will notify the maintainer of the catalog so he can add it to the catalog.

After having discussed this idea at EclipseCon with Domenic, I was quite happy to see it live. Now I'm curious to know how much contortion they had to do :)

Linux Tracing and monitoring

Yet another C focused tool that just amazed me. In this case, the linux tracing and monitoring tools in Eclipse from the linux tools project was showing how various linux tracing and monitoring tools were integrated in the IDE. I really appreciated the efforts made on the visualization of the data (various charts, searching, etc.) and how scalable and snappy was the tool (it literally needs to be able to handle millions of events). If you are showing large data sets in SWT table, there is probably some code in there that you want to look at.

Kudos to the developers.

Tycho, building OSGi bundles with Maven

I gave a very quick demo of building bundles with Maven. This was a condensed version of the webinar I did a couple weeks ago.

Natural Language Processing for Eclipse - The Semantic Assistants

Ninus, from Concordia University, described the usage of natural language processing techniques to help on the day to day development tasks. He showed analysis of the javadoc, and also mentioned some tools that are able to correlate commits with bugs.

Something to keep an eye on, especially to mine old codebases.

Java7 in JDT

Olivier Thomann, last minute guest star at the camp, literally gave a 5 min tour of the new features of Java 7, and gave a status update on the Java 7 support in JDT. Basically most of the support is in place in JDT, but because Java7 will not be released by the end of June, Indigo will not have support for Java 7. *However* Java 7 support will be added to Indigo SR1 that is scheduled for September. In the interim develop builds will be made available.

Montreal demo camp sold out!!!

I just can't believe this. The Montreal demo camp is sold out (45 ppl) ! And the line up of presentation is really looking good. I'm definitely excited to be attending.

Now off to dust off my Quebecois :)

Demo camp Montreal

The Montreal demo camp is fast approaching! I'm sure it will be great event. Don't forget to sign up.

2012 Eclipse annual release revealed

The 2012 Eclipse Annual release will be called Juno. This is still pending the approval of the EMO.

Naming Indigo +1, final vote

It is now time to vote on the name of the 2012 release. The poll will stay open until Sunday night and we will reveal the winning name at EclipseCon.

http://www.eclipse.org/indigo/planning/poll2012name.php

p2 at EclipseCon 2011

This year again, there will be plenty of opportunity to learn and discuss p2 at EclipseCon.

There are four talks:

* p2, your savior or your achilles heel? Everything an Eclipse team needs to know about p2

* Discovering the p2 API's

* Updates in the micro space

* Raising the Eclipse Provisioning to the Clouds

One BOF:

* Fireside chat on p2

And a lot of p2 committers and enthusiasts ready to discuss anything :)

See you there!

Naming Indigo +1

Yes, it is the time of the year where we are looking for a name for the release that will follow Indigo. Actually this year we are doing it earlier with the goal of revealing the name at EclipseCon (or shortly before) in order to facilitate the discussions around your plans so you don't have to keep talking about "Indigo +1" or "the release that will come after Indigo", but instead you can just use the name.

We will be gathering name ideas until early March in Bug 336942.