\section{Related Work}
There have been several works on depression detection over the years. Text-based approaches using classical machine learning algorithms\cite{burdisso2019text,yazdavar2017semi,shen2018cross,kumar2019anxious} mostly used number of tweets and social interaction as network features. Different linguistic features such as profile bio, comments and status data were utilized from social media platforms for user-level depression detection. However, due to their limitations they failed to generalize in most of the unseen cases. In recent years, many deep learning algorithms\cite{rao2020mgl,trotzek2018utilizing,uban2022explainability,alhuzali2021predicting} were able to achieve state-of-the-art results across different data modalities. Rao~\textit{et al.}\cite{rao2020mgl} proposed a Multi-Gated LeakyReLU CNN (MGL-CNN) for identifying depressed individuals in online forums using list of words and posts. Trotzek~\textit{et al.}\cite{trotzek2018utilizing} developed a convolutional neural network based on different word embeddings using ensemble learning. Uban~\textit{et al.}\cite{uban2022explainability} developed a correlation between depressed users and images containing different animals.

\begin{figure*}[!htb]
    \centering
        \includegraphics[width=16cm,height=5cm]{tcn_2.pdf}
        % \caption*{} % Uncomment and use if you need a sub-caption without a label
    \caption{Attention maps from each encoder are used for model interpretabilty}
    \label{fig:fig2}
\end{figure*}

Recently, multi-modal architectures and vision language models \cite{vedula2017emotional,lin2020sensemood,mann2020see,safa2022automatic,xu2020inferring, hadi2023survey} have become extremely popular, leveraging both text and image features and are able to achieve superior results compared to prior baseline approaches. Lin~\textit{et al.}\cite{devlin2018bert}proposed a CNN-based architecture using multi-modal features by utilizing Bidirectional Encoder Representations from Transformers (BERT) for textual features and CNN for visual features. Gui~\textit{et al.}\cite{gui2019cooperative} introduced a novel cooperative multi-agent model based on text and image features using a reinforcement learning based approach. Fang~\textit{et al.}\cite{cong2018xa} used a Bi-LSTM with attention mechanism to learn visual feature and rich text feature, respectively. However, these approaches ignore temporal cues such as posting time or assume that the posts were posted in a synchronous manner. Stankevich~\textit{et al.}\cite{stankevich2018feature} and Choudhury~\textit{et al.}\cite{de2013predicting} demonstrate that depression symptoms tends to higher at late night interval, highlighting the importance of temporal information such as posting time for depression detection output.

Although, there has been some recent works incorporating time component such as Sawhney~\textit{et al.}\cite{sawhney2020time} introduced a STATENet, a time-aware transformer incorporating historical contexts of tweets as well. Bucur ~\textit{et al.}\cite{bucur2023s} proposed a time-enriched multimodal transformer architecture for detecting depression from social media posts using time2vec positional embeddings. However, these approaches do not effectively handle temporal dependencies. Time-enriched multimodal transformer handles sequential processing of user posts by treating the user timeline as a "bag-of-posts” which does not accurately capture temporal information. Similarly, these approaches lack model explainability thus limiting user understanding of how different features are contributing to final prediction of depression detection.

To address the above limitations we introduce Multi-Explainable TemporalNet (MTEN). A TCN based multi-modal transformer which leverages temporal cues such as relative time stamps, along with image and text embeddings for enhanced user-level depression detection. Through self and cross attention mechanism it allows to dynamically adapt the fusion of weights for different modalities. We utilize attention maps for each modality for model interpretability, helping users to understand key features leading to depression. 






