Abstract:
In the current changing trends, collaboration between di erent organizations or consolidation
between applications of the same organization has become a common phenomenon. In order to
achieve strategic business objectives it is necessary to have a uni ed view of data, which is given
by Enterprise Data integration (DI). Based on the requirements of di erent organizations, large
number of tools and technologies are available in the market. Some organizations are already
using di erent integration techniques from past many years. However due to the emerging era
of big data and cloud enterprises want to shift from their old integration methods to the new
and advance techniques.
In this dissertation, a framework has been developed to retrieve the structure of the enterprises
integration repositories and present them visually, so that the enterprises can take completely
informed decisions, as they cannot change what they do not understand. This framework can
be used to nd the connection link information, connection location and the frequency of the
repetition of the same sources and links. This tool can also be used to nd the amount of
data transferred from one geographical area to the other, which will help the organizations in
measuring the bandwidth requirements across networks. We conducted extensive experimental
study on the available datasets of di erent organisations and found that approximately 90% of
sources and 80% of connections are repeated in an integration environment. We discovered the
main reason behind this repetition is the end-to-end connectivity between the creators to the
consumers. To the best of our knowledge, our proposed framework is a unique tool of this type.
We have also implemented a Log Stitching utility, which can stitch the logs of a speci c duration
from various remote and local locations into one le that will help the existing integration tools
in faster debugging of their errors which will indirectly reduce the down time for the applications.
To achieve this, we have designed a unique method of k-way merges by using Java NIOs, and
priority queue.
