AbstractCompiledPage, Missing Plugins, and MANIFEST.MF in FP10 and V10.x
(This entry is
cross-posted from frostillic.us)
Since
9.0.1 FP 10, and including V10 because it's largely identical for this
purpose, I've encountered and seen others encountering a couple strange
problems with compiling XPages projects. This is a perfect opporunity for
me to spin a tale about the undergirding frameworks, but I'll start out
with the immediate symptoms and their fixes.
The
Symptoms
There
are three broad categories of problems I've seen:
-
"AbstractCompiledPage cannot be resolved to a type"
-
Missing third-party XPages libraries, such as ODA, resulting in messages
like "The import org.openntf cannot be resolved"
-
Complaints about MANIFEST.MF, like "MANIFEST.MF has no main section"
and others
The first two are usually directly related and have the same fix, while
the second can also be caused by some other sources, and the last one is
entirely distinct.
Fix
#1: The Target Platform
For
the first two are based on problems in the active Target Platform, namely
one or both of the standard platform components go missing. The upshot
is that you want your Target Platform preferences to look something like
this:
There
should be a selected platform (the name doesn't matter, but "Running
Platform" is the default name) with entries at least for ${eclipse_home}
and for a directory inside your Notes data dir, here C:\Notes\Data\workspace\applications\eclipse.
If they're missing, modify an existing platform or create a new one and
add an "Installation"-type entry for ${eclipse_home} and a "Directory"-type
one for the eclipse directory within your data dir.
Fix
#2: Broken Plugins, Particularly ODA
Though
V10 didn't change much when it comes to XPages, there are a few small differences.
One in particular bit ODA: we had a dependency on the com.ibm.domino.commons
plugin, which was in the standard Notes environment previously but is not
as of V10 (though it's still present on the server). We fixed that one
in the V10 release, and so you should update your ODA version if you hit
this trouble. I don't think I've seen other plugins with this issue in
the V10 transition, but it's a possibility if Fix #1 doesn't do it.
Fix
#3: MANIFEST.MF
This
one barely qualifies as a "fix", but it worked for me: if you
see Designer complaining about MANIFEST.MF, you can usually beat it into
submission by cleaning/rebuilding the project in question. The trouble
is that Designer is, for some reason, skipping a step of the XPages compilation
process, and cleaning usually kicks it into gear.
I've
also seen others have success by deleting the error entry in the Errors
view (which is actually a thing that you can do) and never seeing it again.
I suspect that the real fix here is the same as above: during the next
build, Designer creates the file properly and it goes away on its own.
The
Causes
So
what are the sources of these problems? The root reason is that Designer
is a sprawling mountain of code, built on ancient frameworks and maintained
by a diminished development team, but the immediate causes have to do with
OSGi.
The
first type of trouble - the target platform - most likely has to do with
a change in the way Eclipse manages target platforms (look at the same
prefs screen in 9.0.1 stock and you'll see it's quite different), and I
suspect that there's a bug in the code that migrates between the two formats,
possibly due to the dramatic age difference in the underlying Eclipse versions.
The
second type of trouble - the MANIFEST.MF - is due to a behind-the-scenes
switch in how Designer (and maybe the server) handles dependencies in XPages
projects.
Target
Platforms
The
mechanism that OSGi projects - such as XPages applications - use for determining
their dependencies at build time is the notion of a "Target Platform".
The "target" refers to the notion that this is the platform that
is expected to be available at runtime for what you're building - loosely
equivalent to a basic Java classpath. An OSGi project is checked against
this Target Platform to determine which classes are available based on
their bundle names and versions.
This
is distinct from the related concept of a "Running Platform".
Designer, being based on Eclipse, is itself built on and runs using OSGi.
Internally, it uses the same mechanisms that an XPages application does
to determine what plugins it knows about and what services those plugins
provide.
This
distinction has historically been hidden from XPages developers due to
the way the default Target Platform is set up, pointing at the same Running
Platform it's using. So Designer itself has the core XPages plugins running,
and it also exposes them to XPages applications as the Target. Similarly,
the way we install XPages Libraries like ODA is to install them outright
into the Designer Running Platform. This allows Designer to know about
the library service provided, which it uses to populate the list of available
plugins in the Xsp Properties editors.
However,
as our trouble demonstrates, they're not inherently the same thing. In
standalone OSGi development in Eclipse, it's often useful to have a Target
Platform distinct from the Running Platform - such as the XPages environment
for plugins - to ensure that you only depend on plugins that will be available
at runtime. But when the two diverge in Designer, you end up with situations
like this, where Designer-the-application knows about the XPages runtime
and plugins and constructs an XPages project and translates XSP to Java
using them, but then the compilation process with its empty Target Platform
has no idea how to actually compile the generated code.
MANIFEST.MF
I've
mentioned that an OSGi project "determines its dependencies"
out of the Target Platform, but didn't mention the way it does that. The
specific mechanism has changed over time (which is the source of our trouble),
but the idea is that, in addition to the Java classes and resources, an
OSGi bundle (or plugin) has a file that declares the names of the plugins
it needs, including potentially a version range. So, for example, a plugin
might say "I need org.apache.httpcomponents.httpclient at least version
4.5, but not 5.0 or higher". The compiler uses the Target Platform
to find a matching plugin to compile the code, and the runtime environment
(Domino in our case) does the same with its Running Platform when loading.
(Side
note: you can also specify Java packages to include from any plugin instead
of specific plugin names, but Designer does not do that, so it's not important
for this purpose.)
(Other
side note: this distinction comes, I believe, from Eclipse's switch from
its own mechanism to OSGi in its 3.0 release, but I use "OSGi"
to cover the general concept here.)
The
old way to do this was in a file called "plugin.xml". If you
look inside any XPages application in Package Explorer, you'll see this
file and the contents will look something like this:
<?xml version="1.0"
encoding="UTF-8"?>
<?eclipse version="3.0"?>
<plugin class="plugin.Activator"
id="Galatea2dVCC_2fIKSG_dev5csyncagent_nsf"
name="Domino Designer"
provider="TODO"
version="1.0.0">
<requires>
<!--AUTOGEN-START-BUILDER:
Automatically generated by null. Do not modify.-->
<import
plugin="org.eclipse.ui"/>
<import
plugin="org.eclipse.core.runtime"/>
<import
optional="true" plugin="com.ibm.commons"/>
<import
optional="true" plugin="com.ibm.commons.xml"/>
<import
optional="true" plugin="com.ibm.commons.vfs"/>
<import
optional="true" plugin="com.ibm.jscript"/>
<import
optional="true" plugin="com.ibm.designer.runtime.directory"/>
<import
optional="true" plugin="com.ibm.designer.runtime"/>
<import
optional="true" plugin="com.ibm.xsp.core"/>
<import
optional="true" plugin="com.ibm.xsp.extsn"/>
<import
optional="true" plugin="com.ibm.xsp.designer"/>
<import
optional="true" plugin="com.ibm.xsp.domino"/>
<import
optional="true" plugin="com.ibm.notes.java.api"/>
<import
optional="true" plugin="com.ibm.xsp.rcp"/>
<import
optional="true" plugin="org.openntf.domino.xsp"/>
<!--AUTOGEN-END-BUILDER:
End of automatically generated section-->
</requires>
</plugin>
You
can see it here declaring a name for the pseudo-plugin that "is"
the XPages application (oddly, "Domino Designer"), a couple other
metadata bits, and, most importantly, the list of required plugins. This
is the list that Designer historically (and maybe still; it's not clear)
uses to populate the "Plug-in Dependencies" section in the Package
Explorer view. It trawls through the Target Platform, finds a matching
version of each of the named plugins (the latest version, since these have
no specified ranges), adds it to the list, and recursively does the same
for any re-exported dependencies of those plugins. "Re-exported"
isn't exposed here as a concept, but it is a distinction in normal OSGi
plugins.
Designer
derives its starting points here from implicit required libraries in XPages
applications (all those "org.eclipse" and "com.ibm"
ones above) as well as through the special mechanism of XspLibrary extension
contributions from plugins installed in the Running Platform. This is why
a plugin like ODA has to be installed in Designer itself: in the runtime,
it asks its plugins if they have any XspLibrary classes and uses those
to determine the third-party plugin to load. Here, ODA declares that its
library needs org.openntf.domino.xsp, so Designer adds that and its re-exported
dependencies to the Plug-in Dependencies group.
With
its switch to OSGi in the 3.x series circa 2005, most of the functionality
of plugin.xml moved to a file called "META-INF/MANIFEST.MF".
This starkly-named file is a standard part of Java, and OSGi extends it
to include bundle/plugin metadata and dependency declarations. As of 9.0.1
FP10, Designer also generates one of these (or is supposed to) when assembling
the XPages project. For the same project, it looks like this:
Manifest-Version: 1.0
Bundle-ManifestVersion:
2
Bundle-Name: Domino
Designer
Bundle-SymbolicName:
Galatea2dVCC_2fIKSG_dev5csyncagent_nsf;singleton:=true
Bundle-Version: 1.0.0
Bundle-Vendor: TODO
Require-Bundle: org.eclipse.ui,
org.eclipse.core.runtime,
com.ibm.commons,
com.ibm.commons.xml,
com.ibm.commons.vfs,
com.ibm.jscript,
com.ibm.designer.runtime.directory,
com.ibm.designer.runtime,
com.ibm.xsp.core,
com.ibm.xsp.extsn,
com.ibm.xsp.designer,
com.ibm.xsp.domino,
com.ibm.notes.java.api,
com.ibm.xsp.rcp,
org.openntf.domino.xsp
Eclipse-LazyStart:
false
You
can see much of the same information (though oddly not the Activator class)
here, switched to the new format. This matches what you'll work with in
normal OSGi plugins. For Eclipse/Equinox-targeted plugins, like XPages
libraries, plugin.xml still exists, but it's reduced to just declaring
extension points and contributions, and no longer includes dependency or
name information.
Eclipse
had moved to full OSGi by the time of Designer's pre-FP10 basis (2008's
3.4 Ganymede), but XPages's history goes back further, so I guess that
the old-style Eclipse plugin.xml route is a relic of that. For a good while,
Eclipse worked with the older-style plugins without batting an eye. FP10
brought a move to 2016's Eclipse 4.6 Neon, though, and I'm guessing that
Eclipse dropped the backwards compatibility somewhere in the intervening
eight years, so the XPages build process had to be adapted to generate
both the older plugin.xml files for backwards compatibility as well as
the newer MANIFEST.MF files.
I
can't tell what the cause is, but, sometimes, Designer fails to populate
the contents of this file. It might have something to do with the order
of the builders in the internal Eclipse project file or some inner exception
that manifests as an incomplete build. Regardless, doing a project clean
and usually jogs Designer into doing its job.
Conclusion
The
mix of layering a virtual Eclipse project over an NSF, the intricacies
of OSGi, and IBM's general desire to insulate XPages developers from the
black magic behind the scenes leads to any number of opportunities for
bugs like these to crop up. Honestly, it's impressive that the whole things
holds together as well as it does. Even though it doesn't seem like it
to look at the user-visible changes, the framework changes in FP10 were
massive, and it's not at all surprising that things like this would crop
up. It's just a little unfortunate that the fixes are in no way obvious
unless you've been stewing in this stuff for years.
Re-Launching OpenNTF Snippets
In time for IBM Think's
Community Day, we're relaunching our snippets site.
If you haven't
noticed, things have been changing as OpenNTF has more explicitly embraced
a wider community. Yes, we've always covered all of what has been IBM Collaboration
Solutions / HCL Collaboration Workflow Platforms. But those outside Domino
development have been more of a peripheral audience. That has never been
a deliberate or conscious decision, but our core output - "projects"
- have generally been developer-related (although Easy Admin Tools from
Paul Brigden is a welcome and notable exception recently). XSnippets was
developed as a replacement for the old code bin, but was again focused
on developers. And OpenCode4Connections
was also aimed, primarily, at developer-related extensions to Connections.
Well, with the
latest update to our snippets site, we're more explicitly addressing that.
Firstly, we're
renaming the site OpenNTF Snippets. That's a natural step considering the
focus of content is significantly more than XPages and to make the site
more notably OpenNTF. And secondly, the list of languages has significantly
expanded. These are grouped into specific areas:
- XPages
- Notes Client
/ generic development
- JavaScript development
- DQL
- Admin scripts
- ICEC
- Connections
Customizer
We've also made
a complete overhaul of the UI / UX:
- We've modernised
the look and feel to Bootstrap 3
- User interface
has also included some styling on the buttons taken from non-ICS development
frameworks - primary, friendly and danger button styles
- In terms of
user experience, we've changed from views to multi-faceted search based
on key criteria - language, author and generic free text search. The aim
is that this allows quick filtering down to a handful of results
- The login area
has also been updated including the ability to register and reset password
from the login screen
- The "Create
Snippet" button has been hidden behind the login area
- Some outdated
links have been updated
- Social sharing
has been added (thanks Serdar for adding this)
Hopefully you'll
agree this is more modern, more fit for the future and hopefully it will
give developers ideas to give users a better user experience.
Request for CollaborationToday Curators
CollaborationToday
has become a key place to stay updated on all ICS platforms. In an exciting
year for Domino, Connections and Portal we would like to expand the small
team currently curating blog posts. We want to make sure posts are specifically
relevant to our community and provide best value to those using the site.
The effort required is minimal, reviewing and creating an entry with brief
details and publishing. So if you're interested in helping out, please
contact Oliver
Busse or one of the other
board members via social media
or via the OpenNTF
Slack channels.
Part of the improvements we made last
year to minimise the effort required was auto-tweeting of content posted
there to the CollaborationToday
Twitter Stream. Those stopped
running for a while, but have been re-enabled now. These pick up RSS feeds
for particular areas of focus, so if you want to stay updated on specific
areas via a feed reader, you can use URLs in the format of https://collaborationtoday.info/ct.nsf/feed.xsp?filter=appdevweb.
The filter can be found by clicking on one of the categories in the right-hand
menu of CollaborationToday.
If there are other HCL products that
it would be useful for CollaborationToday to cover blog posts for, and
you have a specific expertise in that area, we are open to adding categories
for those too.
Watch
a short intro to how to moderate articles on CollaborationToday.info here.
New Channels on OpenNTF Slack
Following this week's announcement about
the end of Watson Workspace, the various members of the community asked
for channels relating to IBM Collaboration Solutions / Collaborative Workflow
Platforms technologies to be set up in OpenNTF Slack community. So following
consultation we have channels for Connections on premises, Connections
Cloud, Sametime, Domino admin, Domino dev and IBM Domino Mobile Apps. If
you want to join, you can at https://slackin.openntf.org/
and then access the channels at https://openntf.slack.com.
For XPages there was already a separate
community at https://xpages.slack.com/
and there are no plans to bring that thriving area into OpenNTF channels
at this time. If you're working on XPages and are not yet in that chat,
you can join by going to https://xpages-slack.herokuapp.com/.
If you're a new joiner and new to Slack,
click on the word "Channels" and you get a list of channels you
can choose from. Once you click on one, you're only browsing the content
and will not yet get joined to it. To join the channel, click the link
at the bottom of the chat. Clicking on the star under the bold channel
name at the top of a chat adds it to your Starred channels. We would recommend
everyone to click on their name, go to "Profile & Account"
and set a display name. That ensures an understandable name appears when
you ask questions.
It's great to see more IBM Champions
already joining OpenNTF's slack and I'm sure this will help the breadth
of conversation appearing in the general and random channels, as well as
improving the excellent feedback members can get.
Become a NERD using Domino V10
Nerds are very cool these days - but, you
may ask, what has a NERD to do with Domino V10? Should they not all bleed
yellow? Maybe, but behind the acronym NERD is a very compelling technology
decision! But first, let us jump into the acronym:
N
= Node.js - A nonblocking JavaScript engine based on Chrome's V8 JavaScript
engine
E
= Express - One of the most popular Node libraries, which turns Node into
a REST API / HTML server
R
= React (or, as I prefer, R = Really Cool User Interface) - React is a
client JavaScript framework to build a brilliant user interface
D
= Domino - Yes, Domino: something must be a store for all the data and
even more (i.e. business logic)
Node.js, JavaScript... some of you may
say "finally!", others not. Either way, Node.js is one of the
most sucessful platforms and has with NPM the largest Open Source ecosystem.
And, you may ask, why? Let me quote one of the German Node.js influencers,
Golo Roden (@goloroden): solutions in JavaScript have to be simple and
clean or they will be replaced by simple and clean solutions. Express is
one of the best examples. Two lines of code and you have configured a server
for static HTML files. Two more lines and a REST API is running.
Another popular framework is Passport.
Passport provides integration with a huge variety of different identity
and authentication providers. You want to use Facebook? With Passport,
no problem: done with just a few lines of code.
Node.js is the glue between your front
end and your clever backend services. A lot of integration problems are
already solved by the power of this huge Open Source ecosystem. It makes
complete sense to leverage this power for Domino application. In addition,
with NERD, we have the potential to attract a huge community of JavaScript
developers. We are talking about 4.5 million developers according to LinkedIn.
And it seems that in the year 2018 JavaScript will be the most popular
language!
Dear NERDs let's get ready for domino-db,
the integration point for Node.js. Let's explore a whole new universe of
possibilities!
What can you do until then:
1. Download an IDE that is good for
developing Node.js application. As a Swiss guy, I recommend Visual Studio
Code: https://code.visualstudio.com/
2. Do some baby steps to explore Express:
https://expressjs.com/
3. Stay tuned for my next blog post
Have fun,
Christian
OpenNTF Board of Directors 2018
Following the call
for nominations, the new board has been elected by acclamation.
The Member Directors
have been returned for another two years:
Serdar Basegmez,
Developi Information Systems
Adam Foster, Oval
Jesse Gallagher, I Know Some Guys
Christian Güdemann, Webgate
Douglas Robinson, Prominic
They join the
other Member Directors who still have one year left of their term:
Martin Donnelly
- IBM
Paul Withers -
Intec Systems Ltd
Oliver Busse -
We4IT
Nathan T Freeman
- Red Pill Now
Two of the Contributor
Directors chose to stand again:
Nina Wittich
Fredrik Norling
Padraic Edwards
chose not to stand and we thank him for his work on the board. To replace
him we have Graham Acres of Brytek Systems Inc., IBM Champion and member
of the Cross Canada Collaboration User Group. With Domino V10 just around
the corner, now is an exciting time.
OpenNTF Elections
It's the time of
year when we invite anyone interested in participating in OpenNTF's Board
of Directors to submit their names to ip-manager at openntf.org.
Employees of member
organizations may be nominated as a Member Director – with a two-year
term. There are five such board positions open for election.
Contributors may
be nominated as a Contributor Director – with a one year term. There
are three such board positions open for election.
The following
Member Directors' terms expire this year. They may, of course, run again
for another two year
Serdar Basegmez,
Developi Information Systems
Adam Foster, Oval
Jesse Gallagher,
I Know Some Guys
Christian Güdemann,
Webgate
Douglas Robinson,
Prominic
The terms of the
three Contributor Directors expire this year. Again, they have the
option of running again. They are:
Padraic Edwards
Nina Wittich
Fredrik Norling
Election schedule:
Nominations open
until September 23rd
Circulation of
Candidate Statements September 24th
Voting from September
24th to 29th
The winners take
office on Oct 11th
OpenNTF at ICON UK
This week it's ICON
UK in Birmingham, UK. OpenNTF will again be in attendance. In addition
we will again be sponsoring the beer (and other drinks) at SpeedSponsoring
on Thursday 13th at 5pm. With Domino V10 just around the corner and Connections
Customiser available for a year, now is a great time to get involved in
open source as a contributor or consumer, providing code, reviews, documentation,
or verification on current software versions / platforms. There are plenty
of opportunities to get involved and put open source at the heart of ICS.
So if you're interested in learning more about how you can get involved
or what we're involved in, talk to one of the board members at ICON UK.
Domino V10: Get Involved As A Project Contributor for OpenNTF
With Domino V10 just
around the corner and a commitment already announced at Engage from HCL
and IBM to work with OpenNTF, there is no better time to get involved.
It may seem an onerous task and there may seem more administration than
necessary around contributing to OpenNTF. The measures in place are designed
to give greater confidence to our consumers while minimising the work involved
for contributors. It's actually quick and painless. There are just four
steps:
- Register on
OpenNTF, if you haven't already. If you're not going to be uploading the
releases, you don't even need to do this.
- Sign a CCLA (if you're contributing
on behalf of your company) or ICLA (if you're contributing
as an individual). This just gives you and consumers of your project peace
of mind.
- Add a license
file to your project. Hopefully your project is Apache-licensed, where
possible. The text is here or you can copy
from another project.
- Add a Notice
file outlining who's contributed code and any third-party code you've used..
If you're concerned
about the ICLA / CCLA, I've created a video on our Facebook page to step
through it as well as give an overview of the whole process.
Plus we have our
board member elections in September. Whether on the board or not, we're
always looking for people to get involved in all areas like process management,
infrastructure management etc. The more help we have, the more we can do.
So get involved
and look for more opportunities over the coming months.
OpenNTF Welcomes HCL as a Member Company
At Engage we were
proud to announce that HCL had completed the process to become a member
company of OpenNTF. It has always been great that IBM, as product owners,
supported OpenNTF as a hub and community for open source. And it was no
surprise that HCL were also willing to support OpenNTF's mission. We look
forward to many interesting open source announcements as we move through
Domino 10 and beyond.