Your IP : 216.73.217.79


Current Path : /var/www/cesa.co.za/python/app/services/__pycache__/
Upload File :
Current File : /var/www/cesa.co.za/python/app/services/__pycache__/server_checks.cpython-311.pyc

�

<Ri�S��2�dZddlZddlZddlmZmZmZmZddlmZddl	m
Z
ddlZddlZej
�dee
e��jjj����ddlmZddlmZGd	�d
��Zddeedeeeffd
�ZdS)z�
Server Monitoring Service for IGA Core

This module provides server health monitoring including:
- Memory usage monitoring
- Disk space monitoring  
- CPU load monitoring
- Email alerts when thresholds are exceeded
�N)�Dict�Any�Optional�Tuple)�datetime)�Path)�get_server_monitor_config�)�SymfonyMailServicec���eZdZdZddeefd�Zdeeeffd�Z	deeeffd�Z
deeeffd�Zdeeeffd	�Zd
e
deeeffd�Zdeeeffd
�Zdeeeffd�ZdS)�
ServerMonitorzP
    Server monitoring class that checks system resources and sends alerts.
    N�alert_emailc��t��}|p|d|_t��|_|d|_|d|_|d|_tj��|_tj
��|_dS)z�
        Initialize the server monitor.
        
        Args:
            alert_email: Email address to send alerts to (optional, uses config default if not provided)
        r�disk_threshold�memory_threshold�
cpu_thresholdN)r	rr�mail_servicerrr�psutil�	cpu_count�platform�node�hostname)�selfr�configs   �8/var/www/cesa.co.za/python/app/services/server_checks.py�__init__zServerMonitor.__init__s}��+�,�,��&�?�&��*?���.�0�0���%�%5�6��� &�'9� :���#�O�4��� �)�+�+��� �
����
�
�
��returnc��	g}g}tj��D�]6}	tj|j��}|j|jzdz}d|z
}|j|j|jt|jdzd��t|jdzd��t|j	dzd��t|d��t|d��d�}|�
|��||jkrM|j�d��s3d|j�d|d�d	|d
d�d�}|�
dd
||d�����#t$rY��(t$rY��4wxYwd||tj�����d�S#t$r@}	dt%|	��tj�����d�cYd}	~	Sd}	~	wwxYw)z�
        Check disk usage for all mounted filesystems.
        
        Returns:
            Dict containing disk usage information and alerts
        �d�@�)�device�
mountpoint�
filesystem�total_gb�used_gb�free_gb�percent_used�percent_freez/snapzDISK ALERT: z is �.1fz% full (r(�GB free)�disk�high��type�severity�message�data�success)�status�	disk_info�alerts�	timestamp�error�r5r9r8N)r�disk_partitions�
disk_usager$�used�totalr#�fstype�round�free�appendr�
startswith�PermissionError�	Exceptionr�now�	isoformat�str)
rr6r7�	partition�usager)r*�	disk_data�	alert_msg�es
          r�check_disk_usagezServerMonitor.check_disk_usage5s\��5	��I��F�#�3�5�5�#
�#
�	�"�"�-�i�.B�C�C�E�$)�J���$<��#C�L�#&��#5�L�#,�"2�&/�&:�&/�&6�$)�%�+��*A�1�$E�$E�#(���w�)?��#C�#C�#(���w�)?��#C�#C�(-�l�A�(>�(>�(-�l�A�(>�(>�	!�	!�I��$�$�Y�/�/�/�$�d�&9�9�9�)�BV�Ba�Ba�bi�Bj�Bj�9�C�9�+?�C�C�\�^�C�C� )�)� 4�B�C�C�C�"��
�
�$*�(.�'0�$-�	'�'������'�����D� �����D�����$�&� �%�\�^�^�5�5�7�7�	��
���	�	�	�!��Q���%�\�^�^�5�5�7�7���
�
�
�
�
�
�����	���sM�E;�DD4�2E;�4
E�>E;�	E�
E;�
E�,E;�;
G�5G�:G�Gc��	tj��}t|jdzd��t|jdzd��t|jdzd��t|jd��td|jz
d��d�}g}|d|jkr1d|dd�d|d	d�d
�}|�dd||d
���d||tj
�����d�S#t$r@}dt|��tj
�����d�cYd}~Sd}~wwxYw)z
        Check memory usage.
        
        Returns:
            Dict containing memory usage information and alerts
        r!r"r )r&�available_gbr'r)�percent_availablerQzMEMORY ALERT: Only r+z% memory available (rPr,�memoryr.r/r4)r5�memory_infor7r8r9r:N)r�virtual_memoryr@r>�	availabler=�percentrrBrrFrGrErH)rrR�memory_datar7rLrMs      r�check_memory_usagez ServerMonitor.check_memory_usagess���&	��*�,�,�F�"�&�,�'�":�A�>�>� %�f�&6�'�&B�A� F� F� ����!8�!�<�<� %�f�n�a� 8� 8�%*�3���+?��%C�%C���K��F��.�/�$�2G�G�G�B�+�6I�*J�c�B�B�#�N�3�A�B�B�B���
�
�$� &�(�'�	�����$�*� �%�\�^�^�5�5�7�7�	��
���	�	�	�!��Q���%�\�^�^�5�5�7�7���
�
�
�
�
�
�����	���s�C3C6�6
E�5D;�5E�;Ec
�B�	tjd���}tj��}|d|jz}|d|jz}|d|jz}t|d��t|dd��t|dd��t|dd��t|d��t|d��t|d��|jd�}g}||jkr?d|d�d|dd�d	|dd�d
|dd�d�	}|�dd
||d���d||tj	���
��d�S#t$r@}	dt|	��tj	���
��d�cYd}	~	Sd}	~	wwxYw)z�
        Check CPU usage and load average.
        
        Returns:
            Dict containing CPU usage information and alerts
        r
)�intervalrr")�cpu_percent�	load_1min�	load_5min�
load_15min�load_per_core_1min�load_per_core_5min�load_per_core_15minrzCPU ALERT: Load average is z.2fz per core (1min: z, 5min: z	, 15min: �)�cpu�mediumr/r4)r5�cpu_infor7r8r9r:N)
rr[�os�
getloadavgrr@rrBrrFrGrErH)
rr[�load_avgr_r`ra�cpu_datar7rLrMs
          r�check_cpu_usagezServerMonitor.check_cpu_usage�s��2	� �,�a�8�8�8�K��}���H�"*�!��t�~�!=��!)�!��t�~�!=��"*�1�+���">�� %�[�!�4�4�"�8�A�;��2�2�"�8�A�;��2�2�#�H�Q�K��3�3�&+�,>��&B�&B�&+�,>��&B�&B�',�-@�!�'D�'D�!�^�	�	�H��F�"�D�$6�6�6�d�2D�T�d�d�&�q�k�c�d�d�7?��{�c�d�d�QY�Z[�Q\�c�d�d�d���
�
�!� (�(�$�	�����$�$� �%�\�^�^�5�5�7�7�	��
���	�	�	�!��Q���%�\�^�^�5�5�7�7���
�
�
�
�
�
�����	���s�EE�
F�5F�F�Fc��|jtj�����id�}|���|dd<|���|dd<|���|dd<g}g}|d���D]l\}}|�d��dkr d|vr|�	|d���>|�d��d	kr|�
|���mt|��|d
<||d<||d<|s|r*td|�d
|����|�
|��|S)z{
        Run all system checks.
        
        Returns:
            Dict containing all check results and alerts
        )rr8�checksrlr-rRrcr5r4r7r9�total_alerts�
failed_checksz0[DEBUG] Preparing to send email report. Alerts: �, Failed checks: )rrrFrGrNrXrj�items�get�extendrB�len�print�_send_comprehensive_report)r�results�
all_alertsrn�
check_name�check_results      r�run_all_checkszServerMonitor.run_all_checks�s����
�!����1�1�3�3��
�
��%)�$9�$9�$;�$;����&�!�&*�&=�&=�&?�&?����(�#�#'�#7�#7�#9�#9����%� ��
��
�(/��(9�(?�(?�(A�(A�	1�	1�$�J������)�)�Y�6�6�8�|�;S�;S��!�!�,�x�"8�9�9�9�9��!�!�(�+�+�w�6�6��$�$�Z�0�0�0��"%�j�/�/����&����#0��� �
�	5��	5��q�Z�q�q�bo�q�q�r�r�r��+�+�G�4�4�4��rr7rvc	���	i}|D].}|d}||vrg||<||�|���/d|j�dt|���d�}d|j�dtj���d���dt|���d	�}d
|j�dtj���d���dt|���d
�}|���D]�\}}	|d|����d�z
}|d|����d�z
}|	D]�}|d|d�d�z
}|d|d�d|d�d�z
}d|vrT|d���D]9\}
}t|ttf��r|d|
�d|�d�z
}|d|
�d|�d�z
}�:|dz
}����|dz
}|dz
}|d���D]X\}}
|
�d ��d!kr||�d"�z
}|d#|�d$�z
}�0||�d%|
�d&d'���d�z
}|d#|�d(�z
}�Y|d)z
}|j�
|j||���|�*��d+S#t $rYd+SwxYw),z�
        Send email alerts for all detected issues.
        
        Args:
            alerts: List of alert dictionaries
            results: Complete check results
        r0�SERVER ALERT: � - z issues detected�
Server Health Alert - �
Generated: �%Y-%m-%d %H:%M:%Sz
Total Issues: �

u�
<html>
<head>
    <style>
        body { font-family: Arial, sans-serif; margin: 20px; }
        .header { background-color: #f0f0f0; padding: 15px; border-radius: 5px; }
        .alert { margin: 10px 0; padding: 10px; border-left: 4px solid #ff4444; background-color: #fff5f5; }
        .alert.high { border-left-color: #ff4444; }
        .alert.medium { border-left-color: #ffaa00; }
        .summary { background-color: #e8f5e8; padding: 15px; border-radius: 5px; margin: 20px 0; }
        .metric { margin: 5px 0; }
    </style>
</head>
<body>
    <div class="header">
        <h2>🚨 Server Health Alert</h2>
        <p><strong>Server:</strong> �,</p>
        <p><strong>Generated:</strong> z/</p>
        <p><strong>Total Issues:</strong> �</p>
    </div>
�
�	 ALERTS:
z<h3>z Alerts</h3>�- r2�
    <div class="alert r1�">
        <strong>�
</strong>
r3�  �: �<div class='metric'>�</div>z
SYSTEM SUMMARY:
z,<div class='summary'><h3>System Summary</h3>rlr5r4�: OK
�<div class='metric'><strong>z:</strong> OK</div>z
: ERROR - r9�
Unknown errorz:</strong> ERROR</div>z</div></body></html>��to_email�subject�body�	html_bodyN)rBrrsrrF�strftimerp�upper�
isinstance�int�floatrqr�
send_emailr�striprE)rr7rv�alert_groups�alert�
alert_typer��	text_bodyr��type_alerts�key�valuerxrys              r�_send_alertszServerMonitor._send_alertss���W	��L��
7�
7��"�6�]�
��\�1�1�/1�L��,��Z�(�/�/��6�6�6�6�W�t�}�V�V��V���V�V�V�G���}����L�N�N�#�#�$7�8�8����6�{�{����I�� &*�]�!��")1����(?�(?�@S�(T�(T�#��$,/�v�;�;�%���I�.,8�+=�+=�+?�+?�
*�
*�'�
�K��@�*�"2�"2�"4�"4�@�@�@�@�	��D�J�$4�$4�$6�$6�D�D�D�D�	�(�*�*�E��!:�e�I�&6�!:�!:�!:�:�I��"��Z�(�"�"��y�!�"�"�"��I�����*/��-�*=�*=�*?�*?�Y�Y�J�C��)�%�#�u��>�>�Y� )�-B�#�-B�-B��-B�-B�-B� B�	� )�-X�C�-X�-X�5�-X�-X�-X� X�	����)�I�I�*�"
�/�/�I��G�G�I�,3�H�,=�,C�,C�,E�,E�
c�
c�(�
�L��#�#�H�-�-��:�:��J�!6�!6�!6�6�I��!_�
�!_�!_�!_�_�I�I��J�!h�!h�,�:J�:J�7�Tc�:d�:d�!h�!h�!h�h�I��!b�
�!b�!b�!b�b�I�I��/�/�I�
��(�(��)���_�_�&�&�#�	
)�
�
�
�
�
���	�	�	��D�D�	���s�I"I&�&
I4�3I4c
�`
�	|�dd��}|�dg��}td|�dt|������td|j����|j���}td|�d���d	|�d
������td|�d���d
|�d������|dkr"t|��dkrtd��d|j�d|�dt|���d�}d|j�dtj���	d���d|�dt|���d�	}d|j�dtj���	d���d|�dt|���d�	}|rY|dz
}|D]L}|d �|i��}	|	�d!d"��}
|d#|�d$|
�d%�z
}|d&|�d|
�d'�z
}�M|d%z
}|dk�r8|d(z
}|d)z
}i}|�d*g��D].}|d+}
|
|vrg||
<||
�
|���/|���D]�\}
}|d,|
����d-�z
}|d'|
����d.�z
}|D]�}|d/|d0�d'�z
}|d1|�d2d3���d4|d0�d5�z
}d6|vrT|d6���D]9\}}t|ttf��r|d7|�d8|�d'�z
}|d9|�d8|�d%�z
}�:|d%z
}����|d:z
}|d;z
}|d ���D]�\}}	|	�d<��d=kr\|	�d*g��}|r2|d#|�d>t|���d?�z
}||�d@t|���dA�z
}�h|d#|�dB�z
}||�dC�z
}�z|d#|�dD�z
}||�dE�z
}��|d%z
}|dFz
}tdG|j�dH|����tdIt|���dJt|���dK���|j�|j||���|�L��}|rtdM��dPStdN��dPS#t$$rf}tdOt'|��j�d8t+|������ddPl}tdQ��|���YdP}~dPSdP}~wwxYw)Rz�
        Send a comprehensive email report of failed checks and alerts only.
        
        Args:
            results: Complete check results
        rmrrnz9[DEBUG] _send_comprehensive_report called. Total alerts: roz [DEBUG] Alert email configured: z([DEBUG] Mail service config: SMTP Host: �	smtp_hostz, Port: �	smtp_portz#[DEBUG] Mail service config: From: �
from_emailz, Auth required: �
auth_requiredzG[DEBUG] No alerts or failed checks, but continuing anyway (forced send)r|r}z	 alerts, z failed checksr~rr�z
Total Alerts: z
Failed Checks: r�u
<html>
<head>
    <style>
        body { font-family: Arial, sans-serif; margin: 20px; }
        .header { background-color: #f0f0f0; padding: 15px; border-radius: 5px; }
        .alert { margin: 10px 0; padding: 10px; border-left: 4px solid #ff4444; background-color: #fff5f5; }
        .alert.high { border-left-color: #ff4444; }
        .alert.medium { border-left-color: #ffaa00; }
        .summary { background-color: #e8f5e8; padding: 15px; border-radius: 5px; margin: 20px 0; }
        .metric { margin: 5px 0; }
        .check-section { margin: 20px 0; padding: 15px; border: 1px solid #ddd; border-radius: 5px; }
        .check-header { font-weight: bold; margin-bottom: 10px; }
        .failed { border-left-color: #ff4444; background-color: #fff5f5; }
    </style>
</head>
<body>
    <div class="header">
        <h2>🚨 Server Health Alert</h2>
        <p><strong>Server:</strong> r�z/</p>
        <p><strong>Total Alerts:</strong> z0</p>
        <p><strong>Failed Checks:</strong> r�u/<div class='summary'><h3>❌ Failed Checks</h3>rlr9r�r�z:</strong> ERROR - r�zFailed Check: r�u<h3>⚠️ Detected Alerts</h3>z
ALERTS DETECTED:
r7r0z<h4>z Alerts</h4>r�r�r2r�r1rdr�r�r3r�r�r�u8<div class='summary'><h3>📊 System Status Summary</h3>z
SYSTEM STATUS:
r5r4z:</strong> OK (with z alerts)</div>z: OK (with z	 alerts)
u:</strong> ✅ OK</div>r�u:</strong> ❌ ERROR</div>z: ERROR
z</body></html>z[DEBUG] Sending email to z with subject: zEBUG] Email body length: z chars (text), z
 chars (HTML)r�z [DEBUG] Email sent successfully!z@[DEBUG] Email send returned False - email may not have been sentz1[ERROR] Exception in _send_comprehensive_report: Nz[ERROR] Traceback:)rqrtrsrr�
get_configrrrFr�rBrpr�r�r�r�r�r�rEr0�__name__rH�	traceback�	print_exc)rrvrmrn�mail_configr�r�r�rxry�	error_msgr�r�r�r�r�r�r7�
email_sentrMr�s                     rruz(ServerMonitor._send_comprehensive_reportgsm��L	�"�;�;�~�q�9�9�L�#�K�K���<�<�M��B�l�B�B�mp�q~�m�m�B�B�
C�
C�
C��G�T�5E�G�G�H�H�H��+�6�6�8�8�K��B�[�_�_�[�=Y�=Y�B�B�cn�cr�cr�s~�c�c�B�B�
C�
C�
C��K�����8U�8U�K�K�hs�hw�hw�yH�iI�iI�K�K�
L�
L�
L��q� � �S��%7�%7�1�%<�%<��_�`�`�`�s�t�}�r�r��r�r�PS�Ta�Pb�Pb�r�r�r�G���}����L�N�N�#�#�$7�8�8�������M�"�"�	���I��&&*�]�'��()1����(?�(?�@S�(T�(T�)��*,8�+��,-0�
�,>�,>�-���I�6�
&��N�N�	�"/�O�O�J�#*�8�#4�#8�#8��R�#H�#H�L� ,� 0� 0��/� J� J�I��!p�
�!p�!p�_h�!p�!p�!p�p�I��!N�*�!N�!N��!N�!N�!N�N�I�I��X�%�	��a����>�>�	��4�4�	� "��$�[�[��2�6�6�;�;�E�!&�v��J�!��5�5�35��Z�0� ��,�3�3�E�:�:�:�:�0<�/A�/A�/C�/C�.�.�+�J���!H�
�(8�(8�(:�(:�!H�!H�!H�H�I��!D�j�&6�&6�&8�&8�!D�!D�!D�D�I�!,�.�.��!�%>�%�	�*:�%>�%>�%>�>�	�!�&��y�y��X�6�6�&�&��y�!�&�&�&��	�"�U�?�?�.3�F�m�.A�.A�.C�.C�]�]�
��U�#-�e�c�5�\�#B�#B�!]�$-�1F�c�1F�1F�U�1F�1F�1F�$F�I�$-�1\��1\�1\�u�1\�1\�1\�$\�I��!�X�-�	�	�.�"
�S�S�I��-�-�I�,3�H�,=�,C�,C�,E�,E�
:�
:�(�
�L��#�#�H�-�-��:�:�)�-�-�h��;�;�F��;�!�%�J�%�%�dg�hn�do�do�%�%�%��	�!�
�%V�%V�s�6�{�{�%V�%V�%V�V�	�	�!�%g�J�%g�%g�%g�g�	�!�
�%:�%:�%:�:�	�	��!f�
�!f�!f�!f�f�I��J�!9�!9�!9�9�I�I���!�I��)�)�I��X�d�.>�X�X�w�X�X�Y�Y�Y��l��I���l�l�s�S\�~�~�l�l�l�m�m�m��*�5�5��)���_�_�&�&�#�	6���J��
[��9�:�:�:�:�:��Y�Z�Z�Z�Z�Z���	�	�	��b�d�1�g�g�FV�b�b�Z]�^_�Z`�Z`�b�b�c�c�c������'�(�(�(����!�!�!��D�D�D�D�D�����	���s�R(R=�,R=�=
T-�AT(�(T-c��	|jtj��tj��|jt	j��r%t	j�����ndtjt	j	�����
��tj��tjt	j	����z
d�S#t$r}dt|��icYd}~Sd}~wwxYw)zz
        Get general system information.
        
        Returns:
            Dict containing system information
        N)rr�python_versionr�cpu_freq�	boot_time�uptimer9)rrr�rrr��_asdictr�
fromtimestampr�rGrFrErH)rrMs  r�get_system_infozServerMonitor.get_system_info�s���	%� �M�$�-�/�/�"*�"9�";�";�!�^�;A�?�;L�;L�V�F�O�-�-�5�5�7�7�7�RV�%�3�F�4D�4F�4F�G�G�Q�Q�S�S�"�,�.�.�8�+A�&�BR�BT�BT�+U�+U�U���
���	%�	%�	%��S��V�V�$�$�$�$�$�$�$�����	%���s�CC�
D�&C<�6D�<D�N)r��
__module__�__qualname__�__doc__rrHrrrrNrXrjrz�listr�rur��rrr
r
sO��������(�(�H�S�M�(�(�(�(�,<�$�s�C�x�.�<�<�<�<�|-�D��c��N�-�-�-�-�^9��c�3�h��9�9�9�9�v'��S�#�X��'�'�'�'�R_�4�_�$�s�C�x�.�_�_�_�_�BS�$�s�C�x�.�S�S�S�S�j%��c�3�h��%�%�%�%�%�%rr
rrc�H�t|��}|���S)z�
    Quick function to run all server checks.
    
    Args:
        alert_email: Email address to send alerts to (optional, uses config default if not provided)
    
    Returns:
        Dict containing all check results
    )r
rz)r�monitors  r�quick_server_checkr�s#���K�(�(�G��!�!�#�#�#rr�)r�rfr�typingrrrrr�pathlibrr�sys�path�insertrH�__file__�parentrr	�mailrr
r�r�rr�<module>r�sB����
�	�	�	�
�
�
�
�-�-�-�-�-�-�-�-�-�-�-�-������������������
�
�
������3�3�t�t�H�~�~�,�3�:�;�;�<�<�<�,�,�,�,�,�,�$�$�$�$�$�$�t%�t%�t%�t%�t%�t%�t%�t%�p$�$�H�S�M�$�T�#�s�(�^�$�$�$�$�$�$r