05 Dec 2016
This blog is an update about the use of
C++11 and
C++14 (“modern C++”) within
OME Files C++.
We have been evaluating the switch from C++98 to C++11 for over 18
months. Up until now, we have remained using C++98 in order to
continue to support older systems with compilers which do not support
C++11. However, circumstances have recently changed which will
necessitate a switch at some point in the near future.
Most current platforms support C++11 and C++14, with some exceptions
(workarounds are noted)
| Platform |
Compiler |
C++11 |
C++14 |
| CentOS 6 |
GCC 4.6 |
Unsupported by default* |
Unsupported by default* |
| CentOS 7 |
GCC 4.8 |
Mostly supported |
Partially supported |
| FreeBSD 10 |
Clang/LLVM 3.4 |
Fully supported |
Fully supported |
| FreeBSD 11 |
Clang/LLVM 3.8 |
Fully supported |
Fully supported |
| MacOS X 10.10 |
Apple LLVM 7.3 |
Fully supported |
Fully supported |
| MacOS X 10.11 |
Apple LLVM 8.0 |
Fully supported |
Fully supported |
| MacOS X 10.12 |
Apple LLVM 8.0 |
Fully supported |
Fully supported |
| Ubuntu 14.04 |
GCC 4.8 |
Mostly supported |
Partially supported |
| Ubuntu 16.04 |
GCC 5.3 |
Fully supported |
Fully supported† |
| Visual Studio 2013 |
VC12‡ |
Partially supported |
Mostly unsupported |
| Visual Studio 2015 |
VC13‡ |
Supported |
Partially supported |
| Visual Studio 2017 |
VC15‡ |
Supported |
Supported |
* The software collections
devtoolset-4
(GCC 5.2) and
devtoolset-3
(GCC 4.9) packages provide updated compilers which can build OME
Files C++ using C++11
† Boost 1.58 provided by this release does not build OME Files with
C++14, but does build with C++11; see trac
ticket
‡ Microsoft Visual C++ lags significantly behind GCC and Clang/LLVM,
but VS2013 contains a useful subset of features, which are improved
upon significantly with VS2015 and later
The support classifications above are broad; please see the following
references for full feature coverage:
Third-party dependencies
All of our third-party dependencies have supported building with a
C++11 or C++14 compiler for some time. However, more recently some of
our dependencies have begun to require it, or will require it in
their next major release.
For many users, these libraries are provided by package repositories,
including Linux distributions’ package managers, MacOS X homebrew and
FreeBSD ports. As these package repositories begin to provide the
updated C++11 versions of these libraries, OME Files C++ will no
longer build with them. While some of these C++11 libraries are
optional (Qt5), others such as ICU are required by Boost and
Xerces-C++ for full Unicode support, which is needed by the OME-XML
data model. As a result, C++11 support will become a necessity for
continued use of these libraries as the versions requiring C++11 enter
mainstream use.
The benefits of C++11
C++11 adds a number of useful features to the language, including:
auto typedecltype keyworddefaulted and deleted functions- delegating constructors
- forward-declared
enums
- initialiser lists
- lambdas
nullptr keywordoverride and final qualifiers- range-based
for loops
- rvalue references
static_assert- strongly-typed
enums
- variadic templates
and library features, including:
std::arraystd::shared_ptrstd::threadstd::tuplestd::unique_ptr
These features are all supported by the compilers on the above
platforms. Some additional features (not included here) are not yet
supported by the compilers on every platform, and so can’t yet be
used.
These features benefit the project in several ways:
- they reduce the code size dramatically by permitting much more
concise code with reduced boilerplate (by around 50% in one test
conversion of a single component)
- they allow the use of standard library features in place of Boost
equivalents, which aids interoperability (e.g.
std::thread and
std::shared_ptr in place of boost::thread and
boost::shared_ptr)
- they allow improved performance of standard library containers
through the use of rvalue references, and also allow the use of new
features such as
std::unique_ptr to replace boost::shared_ptr in
certain situations, again improving performance
- they make the library easier to use, for example the use of
initialiser lists can reduce the
PixelBuffer nD array addressing
from multiple lines to a single line of code
- they make the library more understandable and more familiar for
users already using these features with other libraries; in
particular features such as
std::shared_ptr have been available
since VS2010 for Windows developers and their use is already
widespread
Proposal
Given that:
- all of our supported platforms can support a sizeable subset of
C++11
- our dependencies will increasingly require C++11
- there are significant benefits to switching to C++11 as a result of
the features and performance gains it brings
we plan to make a C++11 compiler a requirement in our upcoming 0.3.0
release of OME Files C++, and begin using a subset of the C++11
features available across our supported platforms. For most users,
the transition should be entirely transparent and will require no
immediate code changes as we will leave compatibility typedefs in
place through a transitional period.
If this will present any problems, please get in touch through our
mailing
lists or
forums so that we can
discuss any concerns you may have. Likewise, if the change is
something which you will find useful and beneficial, we would also
like to hear from you.
01 Nov 2016
This blog is an update about what we are working on in the OME Files and
Bio-Formats codebase for the next few months.
Following our Data Model post,
we spent the first semester of 2016 working on a new version of the OME Data
Model which was released in June 2016.
This schema release was then followed by two releases in August 2016:
- the release of our reference implementation for the OME file formats,
OME Files C++ 0.2.0
with full support for all versions of the OME Data Model
- the release of Bio-Formats 5.2.0
including upgrade to the 2016-06 model, API additions and bug fixes.
Since August, a series of patch releases have been delivered both for OME
Files C++ 0.2.x and Bio-Formats 5.2.x. These releases extended the
C++ reference implementation to support all modalities of OME-TIFF allowed by
our format specification, and also provided a series of bug fixes for the
community following a major release.
In addition, we also performed a minor release
of the OME Data Model extending the AcquisitionMode enumeration
to support new values.
Since the beginning of October 2016, our focus has switched to the development
of Bio-Formats 5.3.0. This release includes two major parts: a re-architecture
of the components and new backwards-compatible API additions.
As part of the development of OME Files, a large part of the C++ code
constituting the reference implementation has been migrated out of the
Bio-Formats repository into its own set of components.
In Bio-Formats 5.3.0, we will be applying a similar re-architecture effort for
our Java codebase and the OME Data Model. While Bio-Formats has been
historically a large multi-module project, the idea here is to facilitate
development and shorten build time, provide a better separation of the
functional components and allow faster releases of the low-level components.
In particular the following components have been filtered out to their own
repository and will be managed separately:
All decoupled components have been released and uploaded to the
Maven Central repository under the
org.openmicroscopy groupId.
Bio-Formats 5.3.0 will also include API additions. In order to minimize the
impact of these changes, the codebase is migrating towards
semantic versioning. This means that all the API added
in 5.3.0 will be fully backwards-compatible and should require no changes for
consumers using Bio-Formats 5.2.0. Two main parts of the API will be reviewed
and extended in 5.3.0:
Timeline
We hope to release Bio-Formats version 5.3.0 at the beginning of December 2016.
You can follow our progress on the Trello board.
31 Aug 2016
The challenges and cost associated with the development and maintenance of
software for reading images stored in proprietary file formats (PFFs) have
been discussed at length in previous blog posts
1,
2.
One approach to help address these issues is the development of community
collaborations that provide sustainable solutions to PFF support in
Bio-Formats.
In this blog post, we want to describe two successful examples of partnerships
established with well-recognized commercial entities,
Carl Zeiss Microscopy GmbH and
Intelligent Imaging Innovations (“3i”)
and how this will result in more open, reusable code and better tools for the
imaging community.
ZEISS: Open support for JPEG-XR compression
Some image acquisition systems built by Carl Zeiss Microscopy GmbH store
binary image data using JPEG-XR compression.
Open-source efforts to decode data stored using this technology have to date
been unsuccessful. As we have noted previously, the complexity of providing a
pure Java implementation of this compression scheme was simply too high to be
fully assumed by a non-profit, grant-funded organization like OME.
In response, the user community has raised their concerns to ZEISS and a
partnership has been established with
Glencoe Software
to add a Java-based JPEG-XR decoder to Bio-Formats. Thanks to extensive,
fruitful discussions with ZEISS all outputs of this partnership are publicly
available and all source code licensed identically to other OME projects. Some
public examples of this ongoing work are:
In addition to being a powerful example of how commercial partners can work
together, its philosophical implications should not be understated. In
particular, the openness of the result demonstrates a cultural shift in the
commercial instrumentation community towards the appreciation of
community-based open-source projects.
3i: Maintaining a native reader
3i designs and manufactures systems for biological imaging, powered by their
SlideBook software. Maintaining Bio-Formats support alongside the fast
development of the SlideBook format and their multiple versions has also
proven to be challenging. Over the last five years, developers in the 3i team
have been building contact with the Bio-Formats development team to implement
a native solution to read SlideBook image formats.
A first version of a native Slidebook reader for 32-bit Windows was integrated
and released as part of
Bio-Formats 5.1.2.
Following discussions with members of the 3i team who attended the 2016 OME
Users Meeting, this reader has been separated from the Bio-Formats source code
and is completely maintained by the 3i team. As of Bio-Formats 5.2.0,
the 3i reader became fully pluggable into Bio-Formats e.g. via ImageJ/Fiji.
The 3i library is the first Bio-Formats reader where the development, quality
control and release processes are fully managed by a third-party.
Support has also been added for Linux and Mac OS X platforms with 32 and 64 bit architectures. Integration into a platform like OMERO is an obvious next step.
What’s next
Collaboration and integration with different organisations undoubtedly comes
at some cost: there are differences in objectives, priorities, process and
culture to reconcile. However, our experience in these partnerships has been
uniformly positive and the collaborations have been efficient and productive.
We believe one under-appreciated aspect of open-source development is the
ability to adapt to different scenarios and requirements, in order to achieve
outcomes that benefit the whole community. We look forward to growing these
kinds of collaborations for the community’s benefit.
23 Aug 2016
This is the first of what will hopefully be a series of posts about tools
developed by members of the community. Ian Munro of Imperial writes:
Over the course of adding OMERO capabilities to a number of pre-existing
software tools, I felt that there was a need for a lightweight graphical tool
that would make it possible to easily add the capability of allowing users to
select images from OMERO, to desktop tools.
What I needed was an OMERO equivalent of JFileChooser or Matlab’s uigetfile
or Qt’s QFilelDialog.
OMEROImageChooser is the result.
It is a graphical tool which allows a variety of OMERO objects (Images,
Datasets, Plates, Attachments) to be selected with a look and feel modelled on
OMERO.insight. It can easily be called from Java or Matlab code and is
presented as a quick and relatively easy path to either developing a new OMERO
desktop client or adding an OMERO interface to existing code.
The source is available from https://github.com/imperial-photonics/omeUiUtils.
A ready-to-use .jar file compatible with OMERO 5.2.x can be also downloaded
from https://bintray.com/imperial-photonics/omeUiUtils/omeUiUtils/.
For instructions on how to integrate with Maven/Gradle, refer to the
Bintray user documentation.
There are a number of options available, depending what feedback is needed
from the user. The screenshots below illustrate some of these.
-
Selecting one or more images:
-
Selecting an image or attachment:
-
Selecting a plate:
-
Selecting a Dataset and file name for importing files to OMERO - e.g for
use from acquisition tools:
05 Jul 2016
The recent EU referendum will undoubtedly have implications for the future of
UK science. It now seems equally clear that those implications will remain
unknown for some time to come. No matter what happens next, we want to
emphasize our ongoing commitment to the OME project and to our community.
It is unlikely that our current funding will be altered by the outcome of the
referendum or by the apparent upcoming changes to the political landscape. The
major awards that support OME’s work–from the Wellcome Trust and BBSRC–are
from UK-based institutions. We also participate in several EU projects–
Euro-BioImaging,
Global BioImaging,
MULTIMOT, and
CORBEL–that will
expire within the two-year window being discussed in the media as the minimum
time needed to negotiate a UK departure from the EU. Our understanding is that
our participation in these projects is not in question. (We also have newly
awarded funding to announce–once we have the final documentation!)
Furthermore, our recent Users Meeting in Dundee
once again confirmed strong support from our community and the critical need
to develop the tools that will enable the discoveries from next generation
imaging technologies that are now appearing in the community’s laboratories.
None of our immediate milestones will be modified, and we will keep delivering
Bio-Formats,
OMERO, and the
Image Data Resource just as our users
expect. We will continue to explore new avenues for funding, based on our
community’s need and validation of our software, just as we have done since
2002.
Since the project’s origins, our identity and values have been rooted in
openness, and international, cross-cultural collaboration. OME’s achievements
have always been made possible by the combined talents of an international
team, dedicated to open source software, open development practice, and open
engagement with our broad, diverse, and dedicated user community. Today, the
members of our development team represent more than ten different nations from
both within and outside the EU. Many of us are “immigrants”–we’ve actively
chosen to live in new countries with our families to deliver on the promises
of interoperability and scale in scientific image informatics. We are a
dedicated, multi-national, multi-lingual team that builds and delivers tools
that support the global bioimaging community. We think diversity is one of our
strengths, as it ensures a range of viewpoints and ideas. We therefore
reaffirm our support for inclusion and diversity, and strongly object to the
politics of fear and racism that have characterised some of the recent
political discussions, referenda and elections. We are an example of the best
aspects of the modern, interconnected world that unites dedicated, talented
individuals, regardless of race, religion, gender, sexual identity or national
origin, with a mission to drive scientific insight and discovery for the
benefit of all humankind.
