While the benefits of exploiting Contextual Information (CI) are starting being recognized by the Information Fusion (IF) community [1], most current approaches for CI inclusion lead to stove-piped solutions that hardly scale or adapt to new input or situations. This paper makes a step in the direction of better CI exploitation by presenting some results of an international collaboration investigating the role of CI in IF and proposing an adaptive framework that dynamically takes into consideration CI to better support mission goals. In particular, we discuss some architecture concepts to be considered in the development of fusion systems including CI and we present how context can be dynamically exploited at different levels of a fusion engine. The concepts are illustrated in a maritime use-case.