\documentclass[final,letterpaper,12pt,oneside]{class_diss}

\input{packages}
\input{macros}

\doublespacing
\begin{document}

\input{coverpages}

%\input{nomenclature}
\input{acknowledge}
\input{dedication}
%\input{preface}

\newpage
\pagenumbering{arabic}
\setcounter{page}{1}

\cleardoublepage
\chapter{Introduction}\label{chap:intro}

This cumulative dissertation comprises research in diverse areas related to both ontology reasoning and ontology modeling. While these two topics differ in certain important ways, fundamentally data representation and modeling is central to the development of common-sense and useful reasoning methods, and useful reasoning applications enhance and enrich semantic models with ontologies. In this chapter some initial context is provided from a high-level perspective so that readers may familiarize themselves with the general concepts behind the research presented in following chapters containing the academic publications. More specific information will be provided in the chapters themselves that pertains to the content involved. At the end of this chapter some motivation will be presented that provides some justification to bind the chapters together in a centralized way and that will be examined in the the conclusion.

\section{Ontologies}
The word {\em ontology} comes from a domain of philosophy, specifically metaphysics, that attempts to describe what things exist, and investigates the nature of this existence. Ontologies in computer science are called this way because they are used for a similar reason, though in this case the application is more pragmatic. An ontology is, to put it very briefly, a formal artifact that describes in a logically precise manner the way data should be organized in order to to represent the intended semantics of whatever domain it describes. This means that, like this dissertation, research into ontologies has both a formal part that investigates how they can be leveraged for reasoning applications, and also a user-centric part that involves the semantics of what people want to represent and how to facilitate this in an efficient way.

Ontologies are often expressed in the Web Ontology Language (OWL) \cite{horridge2009owl}, and OWL ontologies correspond to particular description logics (DL). Typically a DL has three major components all of which are used to form axioms with a flexible set of logical operators: classes, roles\footnote{Classes are sometimes referred to as concepts, and roles are sometimes referred to as properties. As the terms are equivalent we will prefer the ones shown here.}, and individuals. The type of description logic used can vary based on what logical operators are used in the ontology. 

Typically OWL ontologies support description logics with operators such as conjunction $\sqcap$, disjunction $\sqcup$, existential quantification $\exists$, universal quantification $\forall$, and binary composition $\circ$, all of which are connected with $\sqsubseteq$ to form axioms out of expressions. The choice of which operators to support in a given ontology has significant impact on the semantics for both modeling and reasoning.

\def\arraystretch{1.15}
\setlength\tabcolsep{8pt}
\begin{table*}
	\caption{\ELp\ Semantics}
	\label{tab:sem}
	\begin{center}
		\makebox[\linewidth]{
			\begin{tabular}{ c|c|c } 
				%HEADER
				Description & Expression & Semantics\\ 
				%EL+
				\hline 
				Individual & $ a $ & $ a \in \Delta^{\mathcal{I}} $\\    
				Top & $ \top $ & $ \Delta^{\mathcal{I}} $\\ 
				%Bottom & $ \bot $ & $ \emptyset $ \\
				Concept & $ C $ & $ C^{\mathcal{I}} \subseteq \Delta^{\mathcal{I}} $\\
				Role & $ R $ & $ R^{\mathcal{I}} \subseteq \Delta^{\mathcal{I}} \times \Delta^{\mathcal{I}} $\\
				Conjunction & $ C \sqcap D $ & $ C^{\mathcal{I}} \cap D^{\mathcal{I}} $\\
				Existential Restriction & $ \exists R.C $ & \{ $ a \ | \ $there is $b \in \Delta^{\mathcal{I}}$ such that $ \left( a,b \right) \in R^{\mathcal{I}}$ and $  b \in C^{\mathcal{I}} $ \}\\
				Concept Subsumption& $C \sqsubseteq D$& $ C^{\mathcal{I}} \subseteq D^{\mathcal{I}} $\\				
				Role Subsumption& $R \sqsubseteq S$& $ R^{\mathcal{I}} \subseteq S^{\mathcal{I}} $\\
				Role Chain& $R_1 \circ \cdots \circ R_n \sqsubseteq R$&$ R_1^{\mathcal{I}} \circ \cdots \circ R_n^{\mathcal{I}} \subseteq R^{\mathcal{I}} $\\
				\hline
		\end{tabular}}
		\small{with $\circ$ signifying standard binary composition}\\
	\end{center}	
\end{table*}
\def\arraystretch{1}
\setlength\tabcolsep{6pt}

In Tables \ref{tab:sem} and \ref{tab:alch} we can see the semantics for the two different logics mentioned specifically in this document to illustrate this point. Table \ref{tab:sem} shows the very simple logic of \ELp\ where there is no disjunction, negation, or universal quantification. This makes modeling and reasoning with \ELp\ ontologies quite a bit simpler than ontologies that use \ALCH\ semantics shown in Table \ref{tab:alch} where these connectives do appear and permit more complex expressions. This is not always a linear relationship when comparing description logics however, since we can see that \ELp\ does support $\circ$ while \ALCH\ does not.

\def\arraystretch{1.25}
\begin{table}%[H]
\centering
\caption{\ALCH\ Semantics}
\label{tab:alch}
\begin{tabular}{ c|c|c }
	\multicolumn{1}{c}{Description}&\multicolumn{1}{c}{Expression}&\multicolumn{1}{c}{Semantics}\\
\hline 
	Individual & $ x $ & $ x^{\mathcal{I}} \in \Delta^{\mathcal{I}} $\\    
    Top & $ \top $ & $ \Delta^{\mathcal{I}} $ \\ 
	Bottom & $ \bot $ & $ \emptyset $ \\
	Class & $ B $ & $ B^{\mathcal{I}} \subseteq \Delta^{\mathcal{I}} $ \\
    Role & $ R $ & $ R^{\mathcal{I}} \subseteq \Delta^{\mathcal{I}} \times \Delta^{\mathcal{I}} $ \\
    \hline
    Negation & $ \neg B $ & $ \Delta^I \setminus B^{\mathcal{I}} $ \\
    Conjunction & $ B \sqcap C $ & $ B^{\mathcal{I}} \cap C^{\mathcal{I}} $ \\
    Disjunction & $ B \sqcup C $ & $ B^{\mathcal{I}} \cup C^{\mathcal{I}} $ \\
    Existential Restriction & $ \exists R.B $ & \{ $ x \ | \ $there is $y \in \Delta^{\mathcal{I}}$ such that $ \left( x,y \right) \in R^{\mathcal{I}}$ and $y \in B^{\mathcal{I}} $ \} \\
    Universal Restriction & $ \forall R.B $ & \{ $ x \ | \ $ for all $y \in \Delta^{\mathcal{I}}$ where $ \left( x,y \right) \in R^{\mathcal{I}}$, we have $y \in B^{\mathcal{I}} $ \}\\
    \hline
    Class Assertion&$B(a)$&$\interpI{a} \in \interpI{B}$\\
    Role Assertion&$R(a,b)$&$(a,b) \in \interpI{R}$\\
    Negated Role Assertion&$\neg R(a,b)$&$(a,b) \not\in \interpI{R}$\\
    Class Subsumption& $B \sqsubseteq C$& $ B^{\mathcal{I}} \subseteq C^{\mathcal{I}}$\\
    Class Equivalence& $B \equiv C$ & $ B^{\mathcal{I}} = C^{\mathcal{I}}$\\
    Role Subsumption& $R \sqsubseteq S$& $ R^{\mathcal{I}} \subseteq S^{\mathcal{I}}$\\
    \hline
\end{tabular}
\end{table}
\def\arraystretch{1}

    \subsection{Ontology Reasoning}
    Ontology reasoning is a broad domain that takes the formal axioms and rules present in an ontology and derives new insights from them \cite{baader2003description}. Ontology reasoning is usually deductive: it uses a set of known facts and the schema provided by the ontology with a reasoning algorithm to find out new information. This can be a fairly simple process like we see in part of Chapter \ref{chap:EL+} \cite{DBLP:conf/aaaiss/EberhartEZSH20} where the description logic \ELp\ is used as input to the system, or it can be recursive and complex like the reasoning shown in Chapter \ref{chap:emi} \cite{DBLP:conf/kgswc/EberhartZH23} with the more complicated logic called \ALCH. There are also inductive approaches to ontology reasoning, which attempt to perform or augment reasoning with machine learning and neural network techniques as is shown in of Chapter \ref{chap:EL+} with long short-term memory networks. And of course the backbone of an efficient reasoning algorithm is often the data representation underpinning the algorithm, an example of which is presented Chapter \ref{chap:fowl} \cite{fowl} that provides an interface between the reasoning application and the person who inputs the ontology according to their desired semantics.
    
    \subsection{Ontology Modeling}
    Ontology modeling addresses one of the other major aspects of ontology research, namely the semantics and methodology surrounding their development. This involves laying  groundwork in software like we see in Chapter \ref{chap:fowl}. But it also concerns aspects of usability and design preference as discussed in Chapter \ref{chap:express} \cite{DBLP:conf/esws/EberhartSCSH21}, where an empirical study was undertaken to investigate how complex an ontology really needs to be in order to be useful. This is important for modeling as well as reasoning, since the necessary complexity of an ontology will determine a large portion of the reasoning complexity. In Chapter \ref{chap:uagent} \cite{DBLP:conf/kgswc/EberhartSSHMM20} an example is presented of a particular ontology with examples that shows the results of modeling with a modular methodology that aligns with the observations presented in  Chapter \ref{chap:express}.

\section{Motivation}
Throughout this document there are examples that touch on various aspects of a unified research program. While the separate contributions themselves do not directly mention an overarching theme, it is not hard to detect some themes if a reader knows what to look for. For this reason a short set of related research questions are presented here before the subsequent chapters begin to focus and motivate them. Later in the conclusion in Chapter \ref{chap:conclusion} the questions will be examined in depth.

\begin{figure}[H]
    \centering
    \begin{enumerate}
        \item How can software support both ontology development and modeling as well as reasoning?\label{quest:fowl}
        \item What are novel ways in which ontology reasoning can be efficiently accomplished?\label{quest:emi}
        \item In what ways can ontology reasoning leverage other AI techniques?\label{quest:el+}
        \item How do the semantics of ontology modeling impact ontology reasoning?\label{quest:express}
        \item What does an ontology that was developed with modeling techniques look like?\label{quest:uagent}
    \end{enumerate}
    \caption{Motivating research questions}
    \label{fig:motivationdiss}
\end{figure}

Each of these questions corresponds to one of the five chapters following this introduction, though all questions are probably relevant to some degree throughout. Chapter \ref{chap:fowl} addresses question \ref{quest:fowl}, Chapter \ref{chap:emi} investigates question \ref{quest:emi}, and so on.

\section{Outline}
This cumulative dissertation contains five chapters that each respond to one of the research questions shown in Figure \ref{fig:motivationdiss}. First Chapter \ref{chap:fowl} will present and discuss how an API can support ontology authoring and editing for modeling or reasoning. Next in Chapter \ref{chap:emi} we will see how a reasoner can be developed with the same ontology API as the first chapter that provides an innovative reformulation of a traditional reasoner algorithm with unique benefits and design tradeoffs. The following Chapter \ref{chap:EL+} presents an alternative approach to ontology reasoning that tries to perform symbolic reasoning tasks with a neural network. Then in Chapter \ref{chap:express} we will see an evaluation of how ontology modeling could be done more simply with patterns, and in a way that could greatly impact the reasoning complexity of an ontology. Finally we will see an example of an ontology that was modeled using these patterns.

Each chapter begins with an overview of the publication and a short description of how it connects to the research presented here. Then a duplication of the camera-ready pre-print discussed in the chapter will be reproduced in the appendix for reference. At the end of the document there will be a short conclusion that summarizes the topics discussed and revisits the motivating questions.

\cleardoublepage
\chapter{A Functional API for OWL}\label{chap:fowl}
\section{Overview of Contribution}
This chapter presents work that led to the development of an API for the Web Ontology Language (OWL) called \fowl \cite{fowl}. The \fowl\ API is named in this way because it is implemented in Clojure, a dialect of the Lisp functional programming language, which uses parentheses to encapsulate functions as seen in the logo in Figure \ref{fig:fowl}.

\begin{figure}[H]
    \centering
    \includegraphics[width=0.5\linewidth]{figures/fOWLiconsmall.png}
    \caption{(f OWL) logo}
    \label{fig:fowl}
\end{figure}

\fowl\ provides an alternative style of interaction with ontologies to the commonly used OWL API \cite{10.5555/2890046.2890052} that is implemented in an object oriented style. The existence of alternatives like \fowl\ allows users and developers to interact with OWL ontologies with different styles of programming interfaces. The implementation style also leverages data structures and techniques unique to the language so that OWL ontologies can be interacted with in different ways that can yield positive results in terms of speed and efficiency for the computer and the user.

\section{Connection to Research}
\fowl\ is not by itself a major publication result, it does however connect directly to the subsequent paper where it was used to develop a reasoner. The primary contribution in this work besides its connection to the next chapter is the connecting role it shows between ontology reasoning and ontology modeling and development in response to research question \ref{quest:fowl}. With the API a user can develop ontologies according to their preferred methodology and then later reason with them using applications that interface with the same API.

\cleardoublepage
\chapter{Should I Stay or Should I Go: A New Reasoner for Description Logic}\label{chap:emi}
\section{Overview of Contribution}
The research discussed in the chapter regards a reasoner called Emi that was featured in \cite{DBLP:conf/kgswc/EberhartZH23}. The Emi reasoner is a reformulation of traditional tableau reasoners to solve axioms more like rule-based reasoners. 

\subsection{Tableau Reasoning Algorithms}
A tableau algorithm \cite{baader2003description} is an algorithm that can determine the consistency of an ontology. Usually ontologies that are expressed in logics that contain disjunction, conjunction, universal quantification, existential quantification, and negation, such as $ALC$ and logics that have it as a sub-logic like \ALCH\ in Figure \ref{tab:alch}, require tableau reasoners. 

The tableau algorithm works by drawing out a graph of individuals in the ontology to determine if the membership of the individual can be verified for every class expression. To give an example, assume we have a very simple ontology such as the one shown in Figure \ref{fig:ontfrag}\footnote{In this example A,B,C are class names, R is  a role name, and d,e are individuals.}.  

\begin{figure}[!ht]
    \centering
    \includegraphics[width=0.6\linewidth]{figures/ont.png}
    \caption{Example Ontology Fragment}
    \label{fig:ontfrag}
\end{figure}

Here there are two axioms and three assertions about them, and we can see how a tableau can be constructed in Figure \ref{fig:tabgra}. Each of the two nodes corresponds to a unique individual from the ontology, and the label L contains a set of expressions that the individual is a member of. This simple example is easy enough to trace mentally by simply inspecting the axioms, but a larger ontology with more axioms and individuals can very easily produce a very large graph that is quite complex and interconnected.

\begin{figure}[!ht]
    \centering
    \includegraphics[width=0.5\linewidth]{figures/tab.png}
    \caption{Example Tableau Graph}
    \label{fig:tabgra}
\end{figure}

\subsection{Rule-Based Reasoning Algorithms}
Rule-based reasoners are designed for different types of logic such as Horn-logics and logic programming. These logics often have different semantics than description logics but can often work together when ontologies and rules have compatible expressivity.

A rule-based reasoner comparable to the tableau example seen previously is shown in Figure \ref{fig:datalog}. Here we have the same two axioms and three assertions, they are just written as if they were logic programming axioms. If we want to see how a rule-based reasoner functions, we an simply take the axioms along with all of the assertions we know as facts, and then any time the antecedent of an axiom is true based on the facts that we have we also add the consequent of the axiom. For example, because we know $A(x) :- B(x)$ and we know $A(d)$, then we can also add $B(d)$ to the facts that we know.

\begin{figure}
    \centering
    \includegraphics[width=0.75\linewidth]{figures/datalog.png}
    \caption{Example rules}
    \label{fig:datalog}
\end{figure}

This type of reasoning can also be quite complex as well in more elaborate scenario. However there is a sense in which it can be more intuitive, since at any stage the axiom remains the same and a human often might solve the reasoning problem in a similar way by attempting substitution.

\subsection{Emi}
The Emi reasoner is an attempt to re-imagine the tableau algorithm as a complex type of rule reasoner. This is done by taking the tableau expansion rules such as shown in Figure \ref{tab:tab} \cite{baader2003description} that are used to solve the tableau algorithm and inverting them like we can see in Figure \ref{tab:inv}, so that instead of making a graph with individuals labelled by their expressions, the algorithm instead works with expressions labelled by their individuals. This minor change allows the Emi reasoner to 'solve' axioms as if they were rules by substitution as long as no backtracking is required.

\begin{table}[!ht]
\centering
\begin{tabular}{l r l}
&&the following expansion rules apply to element $x$ when $x$ is not blocked\\
$\sqcap$-rule&if 1.&$(C \sqcap D) \in \mathcal{L}(x)$\\
&2.&$\{C,D\} \not\in \mathcal{L}(x)$\\
&then&$\mathcal{L}(x) \to \mathcal{L}(x) \cup \{C,D\}$\\

$\sqcup$-rule&if 1.&$(C \sqcup D) \in \mathcal{L}(x)$\\
&2.&$\{C,D\} \cap \mathcal{L}(x) = \emptyset$\\
&then&either $\mathcal{L}(x) \to \mathcal{L}(x) \cup \{C\}$\\
&&or else $\mathcal{L}(x) \to \mathcal{L}(x) \cup \{D\}$\\

$\exists$-rule&if 1.&$\exists R.C \in \mathcal{L}(x)$\\
&2.&there is no $y$ s.t. $\mathcal{L}((x,y)) = R$ and $C \in \mathcal{L}(x)$\\
&then&create a new node $y$ and edge $(x,y)$ with $\mathcal{L}(y) = \{C\}$ and $\mathcal{L}((x,y)) = R$\\

$\forall$-rule&if 1.&$\forall R.C \in \mathcal{L}(x)$\\
&2.&there is some $y$ s.t. $\mathcal{L}((x,y)) = R$ and $C \not\in \mathcal{L}(x)$\\
&then&$\mathcal{L}(y) \to \mathcal{L}(y) \cup \{C\}$\\
\end{tabular}%
\caption{\ALC\ Tableau Expansion Rules}\label{tab:tab}
\end{table}

\begin{table}%[H]
\centering
\begin{tabular}{l r l}
$\sqcap$-rule&if 1.&$x \in \mathcal{L}(C \sqcap D)$\\
&2.&$x \not\in \mathcal{L}(C) \cap \mathcal{L}(D)$\\
&then&$\mathcal{L}(C) \to \mathcal{L}(C) \cup \{x\}$\\
&&$\mathcal{L}(D) \to \mathcal{L}(D) \cup \{x\}$\\

$\sqcup$-rule&if 1.&$x \in \mathcal{L}(C \sqcup D)$\\
&2.&$x \not\in \mathcal{L}(C) \cup \mathcal{L}(D)$\\
&then&either $\mathcal{L}(C) \to \mathcal{L}(C) \cup \{x\}$\\
&&or else $\mathcal{L}(D) \to \mathcal{L}(D) \cup \{x\}$\\

$\exists$-rule&if 1.&$x \in \mathcal{L}(\exists R.C)$\\
&2.&there is no $y$ s.t. $(x,y) \in \mathcal{L}(R)$ and $y \in \mathcal{L}(C)$\\
&3.&there is no $z$ s.t. $x$ is blocked by $z$\\
&then&create a new element $y$ with $y \in \mathcal{L}(C)$ and $(x,y) \in \mathcal{L}(R)$\\

$\forall$-rule&if 1.&$x \in \mathcal{L}(\forall R.C)$\\
&2.&there is some $y$ s.t. $(x,y) \in \mathcal{L}(R)$ and $y \not\in \mathcal{L}(C)$\\
&then&$\mathcal{L}(C) \to \mathcal{L}(C) \cup \{x\}$\\
\end{tabular}%
\caption{Inverted \ALC\ Tableau Expansion Rules}\label{tab:inv}
\end{table}

A side effect of this inversion of the tableau is the emergence of many new and potentially powerful optimizations that may not translate to a traditional tableau. One such optimization, called ABox expansion, works lie the rule based reasoners, where a fact that we know for certain from the assertions can cause other new certain facts to emerge from the reasoning process that are a direct result of the assertions. Adding these to the reasoner allows it to vastly reduce the potential search space for a solution by removing impossible configurations from consideration. Another optimization emerges from the need to reformulate the notion of blocking in a functional programming style to ensure termination where there are no global references or variables, which we call local blocking. By maintaining a history where newly created individuals came from, and where their ancestors originated, it is quite straightforward to simply prevent axioms from creating cycles by stopping them from generating new individuals whenever an individual in their history exists as a result of the axiom that is currently being solved. 

\section{Connection to Research}
The Emi reasoner and its accompanying publication demonstrate that there is indeed value in reexamining established reasoning algorithms. Although Emi does not (and likely cannot) challenge fundamentally the proven complexity of reasoning with OWL ontologies, by approaching the problem in a new way it demonstrates that for research question \ref{quest:emi} there are many possible novel methods still available for researchers to explore that can optimize reasoning performance and eplainability.

\cleardoublepage
\chapter{Completion Reasoning Emulation for the Description Logic EL+}\label{chap:EL+}
\section{Overview of Contribution}
This chapter concerns the publication \cite{DBLP:conf/aaaiss/EberhartEZSH20} where a simpler description logic \ELp\ is used to generate input for training a neural network, which is then evaluated to see if it can be used to emulate the behavior or a reasoner by learning the sequences of its inferences.

The inferences that occur in an \ELp\ reasoner are similar in some respects to the rule-based reasoners described in Chapter \ref{chap:emi}. However, because \ELp\ is so simple it can actually be evaluated without inspecting any individuals and instead applying  a series of completion rules like can be seen in Figure \ref{tab:comprul} on axioms themselves. These rules can be used by taking each and searching for axioms in the ontology that fit the pattern on the left side, and then whenever a match is found adding  new inferred axiom to the ontology of the form on the right hand side and repeating the process. For \ELp\ ontlologies there are always a finite number of these entailments that can be generated, and at a certain stage of doing this reasoning every axiom can be checked by every rule and nothing new will emerge. This stopping point is known as completion, hence the name completion rules.

\setlength\tabcolsep{1pt}\def\arraystretch{1.3}
\begin{table}
\caption{\ELp\ Completion Rules}
\label{tab:comprul}	\centering
	\begin{tabular}{|lccccccl|}\hline
		(1)&$A \sqsubseteq C $& &$C \sqsubseteq D$& &&$\models$&$ A \sqsubseteq D$\\
		(2)&$A \sqsubseteq C_1$& &$A \sqsubseteq C_2$& &$C_1 \sqcap C_2 \sqsubseteq D$&$\models$&$A \sqsubseteq D$\\
		(3)&$A \sqsubseteq C$& &$C \sqsubseteq \exists R.D$& &&$\models$&$A \sqsubseteq \exists R.D$\\
		(4)&$A \sqsubseteq \exists R.B$& &$B \sqsubseteq C$& &$\exists R.C \sqsubseteq D$&$\models$&$A \sqsubseteq D$\\
		(5)&$A \sqsubseteq \exists S.D$& &$S \sqsubseteq R$& &&$\models$&$A \sqsubseteq \exists R.D$\\
		(6)&$A \sqsubseteq \exists R_1.C$& &$C \sqsubseteq \exists R_2.D$& &$R_1 \circ R_2 \sqsubseteq R$&$\models$&$A \sqsubseteq \exists R.D$\\\hline
	\end{tabular}	
\end{table}
\def\arraystretch{1}\setlength\tabcolsep{6pt}

Because this reasoning process is deterministic and follows certain pre-defined patterns in terms of the inferences it makes at each stage, the data collected from this reasoner was then fed into a series of different very simple Long Short-Term Memory (LSTM) networks and trained to trace the input ontology into the inferences that should emerge from reasoning with completion rules. The most basic control network can be seen in Figure \ref{fig:flat} where no special help or assistance was given to the network and it was just naively trained as-is with the defined input and output sizes.

\begin{figure}
    \centering
    \includegraphics[width=0.35\textwidth]{old/figures/visualFlat.png}
    \caption{Flat Architecture}
    \label{fig:flat}
\end{figure}

Two other networks were devised that roughly correspond to each other in order to test whether intermediate results from the reasoner can augment or improve the process. Figure \ref{fig:deep} shows a network where the training is shaped according to this intermediate reasoning stage, but which is allowed to train and correct itself from the real inputs and outputs. In Figure \ref{fig:piece} the same network is now split into two parts that are trained separately, first training the input layer to learn intermediate results, then training the output layer to learn answers from intermediate results. On evaluation results are fed from one network directly into the other so that both networks were comparable.

\begin{figure}%[t]
\centering
    \begin{minipage}{0.45\textwidth}
        \centering
        \includegraphics[width=.9\textwidth]{old/figures/visualDeep.png}
        \caption{Deep Architecture}
        \label{fig:deep}
    \end{minipage}
    \begin{minipage}{0.45\textwidth}
        \centering
        \includegraphics[width=.9\textwidth]{old/figures/visualPiecewise.png}
        \caption{Piecewise Architecture}
        \label{fig:piece}
    \end{minipage}
\end{figure}

Although none of these LSTMs achieved groundbreaking accuracy, they did begin to very closely resemble the shape and patterns of  the correct answers, even though the answers themselves were not exact.

\section{Connection to Research}
This chapter relates to the research question \ref{quest:el+} by demonstrating how reasoners can be used with other AI techniques such as LSTMs and neural networks to investigate interesting new ways of performing inference and combining the unique benefits of different styles of AI. The potential to learn with intermediate answers could provide a way to peek inside the black box of many deep learning systems and gain insights into what they are really learning and how this can be leveraged for reasoning purposes.

\cleardoublepage
\chapter{Expressibility of OWL Axioms with Patterns}\label{chap:express}
\section{Overview of Contribution}
The publication \cite{DBLP:conf/esws/EberhartSCSH21} featured in this chapter takes a step away from reasoning directly and investigates how ontologies could be modeled, based on their semantics, and whether this could have been simpler if written differently. This paper is primarily concerning modeling, but reasoning still plays a role as complexity of axioms and how they are authored can impact the performance of a reasoner. From a modeling perspective the complexity of axioms is critical, however, as writing and working with axioms when modeling can have a steep learning curve.

Complexity is managed in the modeling process by adhering to a modular methodology \cite{shimizu2018comodide} that uses patterns to formulate common simple axioms that authors often use when writing ontologies that can also be easily displayed in a visual editor. To evaluate whether existing ontologies {\em could} have been written with these simple patterns (even though they weren't), an empirical analysis was done to see what percentage of the axioms they contained might have emerged from pattern.

This involves first doing a very simple normalization that has no impact on the semantics of the ontology, in case some axioms were written in ways that could very easily be broken down into multiple simple parts. Next these axioms were matched against the axiom patterns shown in Table \ref{tab:owlaxax} in much the same way as the \ELp\ completion reasoner examines axioms, except in this case it is not doing inference but instead counting axioms that were or could have been expressed with each pattern.

\begin{table}[!ht]
\centering
	\begin{tabular}{llp{0.1cm}ll}
    Subclass & \subClass &               &    \begin{tabular}{c}Functional\end{tabular}& \functional \\
    Disjoint Classes & \disjointClasses &    &   \begin{tabular}{c}Qualified Functional\end{tabular}& \qualifiedFunctional \\
    Domain & \domain &            &            \begin{tabular}{c}Scoped Functional\end{tabular}& \scopedFunctional \\ 
    Scoped Domain & \scopedDomain &    &   \begin{tabular}{c}Qualified Scoped Functional\end{tabular}& \qualifiedScopedFunctional \\
    Range & \range &             &            Inverse Functional& \inverseFunctional \\
    Scoped Range & \scopedRange &       &            \begin{tabular}{c}Inverse Qualified Functional\end{tabular} & \inverseQualifiedFunctional \\
    Existential & \existential &     &      \begin{tabular}{c}Inverse Scoped Functional\end{tabular}& \inverseScopedFunctional \\
    Inverse Existential & \inverseExistential &   &  \begin{tabular}{c}Inverse Qualified Scoped Functional\end{tabular}& \inverseQualifiedScopedFunctional \\
     &  &                 &                        Structural Tautology & \structuralTautology 
    \end{tabular}
    \small{A,B,R are variable terms that contain at most one class or role name, or a data range}\par\null\par
\caption{OWLAx Axiom Patterns}	
\label{tab:owlaxax}	
\end{table}

In the resulting evaluation it turns out that, despite performing a very simple analysis, almost all of OWL axioms that occur in OWL ontologies that were inspected could have been or were simple and expressible with patterns.

\section{Connection to Research}
The content in this chapter addresses research question \ref{quest:express} by examining ways in which ontology development methodology that involves simple patterns of tractable axioms can inform and impact a reasoning algorithm. Because we can see that most of OWL doesn't need to be overly complex to still fulfil its purpose, reasoners like Emi that perform well on simple axioms can benefit from a unified design process that encompasses ontology development as well as ontology reasoning.

\cleardoublepage
\chapter{A Domain Ontology for Task Instructions}\label{chap:uagent}
\section{Overview of Contribution}
The final chapter in this document references a publication that uses the methodology \cite{shimizu2018comodide} examined in the previous chapter and produces a tangible artifact.  The ontology itself can be seen in Figure \ref{fig:isrmatb}, as well as the modules that were used to construct it that are indicated by the grey boxes.

\begin{figure}[!ht]
    \centering
    \includegraphics[width=0.9\linewidth]{old/figures/all_draft-6.pdf}
    \caption{ISR-MATB Ontology}
    \label{fig:isrmatb}
\end{figure}

This ontology describes an experiment in cognitive science, where a cognitive agent was designed to perform a set of tasks and respond as a human would. Each task involves some set of instructions, shown in the instruction module, that could describe the agent's environment or situation in the situation description. The agent could then take actions which might alter the environment or experiment. There are also modules to represent affordances the agent had, as well item roles for the items in the agent's environment and of course a module fr the experiment itself.  Each axiom included in the ontology is described in detail in a short axiomatization for each module so that a reader can connect the high-level diagram with the formal axioms.

\section{Connection to Research}
This publication shows an example of what an ontology can look like when it is designed with the methodologies and techniques discussed in the previous chapter. It responds to research question \ref{quest:uagent} by providing such an example and describing the use case in detail to contextualize how the ontology development meets the needs of the users.

\cleardoublepage

\chapter{Conclusion}\label{chap:conclusion}
To tie things together now here in the conclusion, the research questions from Figure \ref{fig:motivationdiss} will be reexamined and discussed in context of their respective chapters.

\begin{figure}
    \centering
    \begin{enumerate}
        \item How can software support both ontology development and modeling as well as reasoning?
        \item What are novel ways in which ontology reasoning can be efficiently accomplished?
        \item In what ways can ontology reasoning leverage other AI techniques?
        \item How do the semantics of ontology modeling impact ontology reasoning?
        \item What does an ontology that was developed with modeling techniques look like?
    \end{enumerate}
    \repeatcaption{fig:motivationdiss}{Motivating research questions}
\end{figure}

\subsection*{Research Question \ref{quest:fowl}:}
\noindent\textbf{How can software support both ontology development and modeling as well as reasoning?}

In Chapter \ref{chap:fowl} an example was given of how software can be developed to represent the semantics of an OWL ontology in efficient ways to meet particular requirements of a user. With \fowl, not only is there a flexible and lightweight API for ontology management in the functional programming paradigm, but also a framework upon which reasoning applications such as Emi can be developed. Although there are certainly other options for different use cases and preferences, \fowl\ represents a strong example of how software can support both ontology development and ontology reasoning.

\subsection*{Research Question \ref{quest:emi}:}
\noindent\textbf{What are novel ways in which ontology reasoning can be efficiently accomplished?}

The Emi reasoner presented in Chapter \ref{chap:emi} demonstrates that new ways of approaching established algorithms an lead to unexpected benefits and insights. Although the Emi algorithm is theoretically related to existing algorithms in terms of complexity, due to the unique implementation it is able to achieve certain benefits that surpass other reasoners in efficiency. This is of course a design trade-off that benefits in some situations and adds complexity in others. In the end it is clear that efficient and novel ways to reason with ontologies are a potentially fruitful topic for further investigation.

\subsection*{Research Question \ref{quest:el+}:}
\noindent\textbf{In what ways can ontology reasoning leverage other AI techniques?}

Chapter \ref{chap:EL+} presents a method whereby an LSTM neural network can be used to emulate a reasoner applying completion rules to an \ELp\ ontology. This method of reasoning trades the certainty of a deductive algorithm for the speed and flexibility of a trained neural network. While the range of possible AI topics that could augment ontology reasoning is quite broad, the research in this chapter provides some useful context into one such attempt to integrate and leverage other AI techniques in ontology reasoning.

\subsection*{Research Question \ref{quest:express}:}
\noindent\textbf{How do the semantics of ontology modeling impact ontology reasoning?}

The study presented in Chapter \ref{chap:express} investigates the extent to which the axiomatic complexity of OWL ontologies is inherent to the language or simply an artifact of design choices and authorship. To a surprising degree it turns out that ontologies are mostly very simple, even with a very naive analysis, and could possibly even be simpler with a more thorough investigation. This hinges on the concept of expressibility, which relates to both ontology modeling and ontology reasoning. If we axioms are expressible simply with patterns, then doing so benefits both the modeler and the algorithm by removing ambiguity and complexity from the statements made.

\subsection*{Research Question \ref{quest:uagent}:}
\noindent\textbf{What does a ontology that was developed with modeling techniques look like?}

In Chapter \ref{chap:uagent} an example was provided that applied the axiom patterns that appeared in Chapter \ref{chap:express} to an actual use case. Here it can be seen how the rather simple statements made in the axiomatization lead to a robust ontology that can represent a range of cognitive science experiments. These simple axioms are used in a modular development methodology, whereby smaller modules are modeled independently and then put together to make a larger ontology. Modularization, in this sense, ensures that the ontology model is sufficiently precise while avoiding undesirable complexity in the modeling process as well as in any potential reasoning applications that may want to be applied in the future.

\cleardoublepage
\phantomsection

\addcontentsline{toc}{chapter}{Bibliography}
\bibdata{references}
\bibliography{references}

\input{appendix}

\end{document}
