403Webshell
Server IP : 93.86.61.54  /  Your IP : 216.73.216.60
Web Server : Apache/2.4.62 (Ubuntu)
System : Linux rasin.ddns.net 6.8.0-124-generic #124~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Tue May 26 21:05:19 UTC x86_64
User : www-data ( 33)
PHP Version : 8.4.22
Disable Function : NONE
MySQL : OFF  |  cURL : ON  |  WGET : ON  |  Perl : ON  |  Python : OFF  |  Sudo : ON  |  Pkexec : ON
Directory :  /usr/share/ibus-table/engine/__pycache__/

Upload File :
current_dir [ Writeable ] document_root [ Writeable ]

 

Command :

[ Back ]     

Current File : /usr/share/ibus-table/engine/__pycache__/tabsqlitedb.cpython-310.pyc
o

��a_�@s�dZddlmZddlmZddlmZddlmZddlmZddlmZddlmZdd	l	Z	dd	l
mZdd	lZdd	l
Z
dd	lZdd	lZdd	lZdd	lZdd	lZdd	lZdd	lZe�d
�Zed�adZdZGd
d�d�ZGdd�d�Zd	S)z7
Module for ibus-table to access the sqlite3 databases
�)�List)�Tuple)�Iterable)�Dict)�Union)�Optional)�CallableNz
ibus-table�1.00u!“”‘’《》〈〉〔〕「」『』【】〖〗（）［］｛｝．。，、；：？！…—·ˉˇ¨々～‖∶＂＇｀｜⒈⒉⒊⒋⒌⒍⒎⒏⒐⒑⒒⒓⒔⒕⒖⒗⒘⒙⒚⒛АБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯЁⅠⅡⅢⅣⅤⅥⅦⅧⅨⅩⅪⅫ⒈⒉⒊⒋⒌⒍⒎⒏⒐⒑⒒⒓⒔⒕⒖⒗⒘⒙⒚⒛㎎㎏㎜㎝㎞㎡㏄㏎㏑㏒㏕ΑΒΓΔΕΖΗΘΙΚΛΜΝΞΟΠΡΣΤΥΦΧΨΩ⑴⑵⑶⑷⑸⑹⑺⑻⑼⑽⑾⑿⒀⒁⒂⒃⒄⒅⒆⒇€＄￠￡￥¤→↑←↓↖↗↘↙ァアィイゥウェエォオカガキギクグケゲコゴサザシジスズセゼソゾタダチヂッツヅテデトドナニヌネノハバパヒビピフブプヘベペホボポマミムメモャヤュユョヨラリルレロヮワヰヱヲンヴヵヶーヽヾぁあぃいぅうぇえぉおかがきぎぱくぐけげこごさざしじすずせぜそぞただちぢっつづてでとどなにぬねのはばひびぴふぶぷへべぺほぼぽまみむめもゃやゅゆょよらりるれろゎわゐゑをん゛゜ゝゞ勹灬冫艹屮辶刂匚阝廾丨虍彐卩钅冂冖宀疒肀丿攵凵犭亻彡饣礻扌氵纟亠囗忄讠衤廴尢夂丶āáǎàōóǒòêēéěèīíǐìǖǘǚǜüūúǔù＋－＜＝＞±×÷∈∏∑∕√∝∞∟∠∣∥∧∨∩∪∫∮∴∵∶∷∽≈≌≒≠≡≤≥≦≧≮≯⊕⊙⊥⊿℃°‰♂♀§№☆★○●◎◇◆□■△▲※〓＃＆＠＼＾＿￣абвгдежзийклмнопрстуфхцчшщъыьэюяёⅰⅱⅲⅳⅴⅵⅶⅷⅸⅹβγδεζηαικλμνξοπρστυφθψω①②③④⑤⑥⑦⑧⑨⑩①②③④⑤⑥⑦⑧⑨⑩㈠㈡㈢㈣㈤㈥㈦㈧㈨㈩㈠㈡㈢㈣㈤㈥㈦㈧㈨㈩ㄅㄆㄇㄈㄉㄊㄋㄌㄍㄎㄏㄐㄑㄒㄓㄔㄕㄖㄗㄘㄙㄧㄨㄩㄚㄛㄜㄝㄞㄟㄠㄡㄢㄣㄤㄥㄦc@s`eZdZdZ		d
deejjdeee	e	fddfdd�Z
de	de	fd	d
�Zde	fdd�ZdS)�
ImePropertiesz=
    A class to cache the properties of an input method.
    N�db�default_properties�returncCsd|duri}|s
dS||_d}z	|�|���}Wn	t�d�Y|D]}|d|j|d<q$dS)ul
        “db” is the handle of the sqlite3 database file obtained by
        sqlite3.connect().
        NzSELECT attr, val FROM main.ime;z'Cannot get ime properties from database�r)�ime_property_cache�execute�fetchall�LOGGER�	exception)�selfrr�sqlstr�results�result�r�+/usr/share/ibus-table/engine/tabsqlitedb.py�__init__Ws�zImeProperties.__init__�keycCs||jvr
|j|SdS)z�
        Return the value for a key from the property cache

        :param key: The key to lookup in the property cache
        �)r)rrrrr�getls

zImeProperties.getcCsdt|j�S)Nzime_property_cache = %s)�reprr�rrrr�__str__vszImeProperties.__str__)NN)
�__name__�
__module__�__qualname__�__doc__r�sqlite3�dbapi2�
Connectionr�strrrr rrrrr
Ss�
��
�
r
c@s�eZdZdZ				d�dededededd	f
d
d�Z					
d�dedededededd	fdd�Zd�dd�Z	d�dd�Z
d�dedd	fdd�Zd�dd�Zd�dd�Z
defdd �Zdefd!d"�Zdefd#d$�Zdefd%d&�Zdefd'd(�Zdedd	fd)d*�Zd+eeeefdd	fd,d-�Zdeeeefeeeeeeffffd.d/�Zdeefd0d1�Zdefd2d3�Zdefd4d5�Z	6d�d7eeeeeefdedd	fd8d9�Z					6	
d�deded:ededededd	fd;d<�Zd=eeeefdd	fd>d?�Z 	6d�d@eeeeefdedd	fdAdB�Z!	6d�dCeeeefdedd	fdDdE�Z"d�dFdG�Z#dHedd	fdIdJ�Z$d�dHedKedd	fdLdM�Z%dede&fdNdO�Z'		P	Qd�dRedSeeeeeefdTedeeeeeeffdUdV�Z(			Q				d�dedWedTedXedYedZed[edeeeeeeffd\d]�Z)		Q		d�dedTedXedYedeeeeeeff
d^d_�Z*	d�d`edeeeeffdadb�Z+d�dcdd�Z,deedd	fdfdg�Z-deede.eeeffdhdi�Z/deedefdjdk�Z0dledefdmdn�Z1dedefdodp�Z2	d�dededefdqdr�Z3d�dededefdsdt�Z4			d�deded[edd	fdudv�Z5dedeefdwdx�Z6				
d�dededededd	f
dydz�Z7d�d{d|�Z8		}d�d~ededeeeeeeffd�d��Z9d	S)��TabSqliteDbu�Phrase database for tables

    The phrases table in the database has columns with the names:

    “id”, “tabkeys”, “phrase”, “freq”, “user_freq”

    There are 2 databases, sysdb, userdb.

    sysdb: System database for the input method, for example something
           like /usr/share/ibus-table/tables/wubi-jidian86.db
           “user_freq” is always 0 in a system database.  “freq”
           is some number in a system database indicating a frequency
           of use of that phrase relative to the other phrases in that
           database.

    user_db: Database on disk where the phrases used or defined by the
           user are stored. “user_freq” is a counter which counts how
           many times that combination of “tabkeys” and “phrase” has
           been used. “freq” is equal to 0 for all combinations of
           “tabkeys” and “phrase” where an entry for that phrase is
           already in the system database which starts with the same
           “tabkeys”.
           For combinations of “tabkeys” and “phrase” which do not exist
           at all in the system database, “freq” is equal to -1 to
           indidated that this is a user defined phrase.
    rF�filename�user_db�create_database�	unit_testr
Nc	Cszttt�d���aWnttfytd�aYnwg|_||_||_	|�
�|s1tj�|j�r9t
�|j�|_ntd|j�z,|j�d�|j�d�|j�d�|j�d�|j�d�|j�d	�|j�d
�Wn	t�d�Y|j�d�id
d�dd�dd�dd�dd�ddt���ddt�d��dd�dd�dd�dd�dd�d d!�d"d#�d$d%�d&d'�d(d)�id*d�d+d,�d-d.�d/d.�d0d.�d1d.�d2d.�d3d.�d4d.�d5d,�d6d7�d8d9�d:d�d;d�d<d=�d>d�d?d,��d,d@dA��|_|�rdB}dC}t|j�D]}||j|dD�}|j�||����s|j�||��qt|j|jdE�|_t|j�d ��|_|j�d�|_ |�!�|_"|�#�|_$|j�d/��%�d,k|_&|�'�|_(|�)�|_*|�+�|_,t�-t.�/�dF�}	tj�0|j��1dGdH�}
t�-|	|
�|_2|�s||�3�|�r�|�r�dS|dIk�r�t�4|	��s�tj�5dJ�}t�4|��r�t�6tj�-|dK�tj7��r�t�8tj�-|dK��t�6tj�-|dL�tj7��r�t�8tj�-|dL��t9�:||	�t9�;|�t�<|	|�ntj=|	dMdN�t�-|	|�}t�>|��s�t�?dO|�n�z�|�@|�}gdP�}
|du�s|dQtAk�s|�B|�tC|
�k�r�t�?dR|�|du�r.t�?dS�|jD|dTdU�|_n4|dQtAk�rIt�?dVtA|dQ�|jD||dQdU�|_n|�B|�tC|
�k�rbt�?dWtC|
�|�B|��g|_t�dX�}t�?dY||�tj�>|��r~t�E|||�tj�>|dZ��r�t�E|dZ|dZ|�tj�>|d[��r�t�E|d[|d[|�t�?d\|�|�F|�t�?d]�nt�?d^|�Wn	t�d_�Yzt�?d`|�|j�da|�Wnct�?db|�t�dX�}t�?dY||�tj�>|��rt�E|||�tj�>|dZ��rt�E|dZ|dZ|�tj�>|d[��r*t�E|d[|d[|�t�?d\|�|�F|�|j�da|�Y|�Gdc�|j�r�g}|jD]}|�H|d|dd|de|dfdg���qLdh}z	|j�I||�Wn	t�di�Y|j�J�|j�dj�|�Kdc�|�L�dS)kN�IBUS_TABLE_DEBUG_LEVELrzCannot open database file %szPRAGMA encoding = "UTF-8";z"PRAGMA case_sensitive_like = true;zPRAGMA page_size = 4096;zPRAGMA cache_size = 20000;zPRAGMA temp_store = MEMORY;z$PRAGMA journal_size_limit = 1000000;zPRAGMA synchronous = NORMAL;z!Error while initializing databasez:CREATE TABLE IF NOT EXISTS main.ime (attr TEXT, val TEXT);�namerz
name.zh_cnz
name.zh_hkz
name.zh_tw�author�somebody�uuidz%s�
serial_numberz%Y%m%d�iconzibus-table.svg�license�LGPL�	languages�language_filter�valid_input_chars�abcdefghijklmnopqrstuvwxyz�max_key_length�4�commit_keys�space�select_keys�1,2,3,4,5,6,7,8,9,0�page_up_keysz!Page_Up,KP_Page_Up,KP_Prior,minus�page_down_keysz$Page_Down,KP_Page_Down,KP_Next,equal�
status_prompt�def_full_width_punct�true�def_full_width_letter�false�user_can_define_phrase�pinyin_mode�suggestion_mode�dynamic_adjust�auto_select�auto_commit�
auto_wildcard�descriptionzA IME under IBus Table�layout�us�symbol�rules�least_commit_length�0�start_chars�orientationz{})�always_show_lookup�char_promptsz9
            SELECT val FROM main.ime WHERE attr = :attr;zC
            INSERT INTO main.ime (attr, val) VALUES (:attr, :val);��attr�val�rr�tables�.dbz.cache�:memory:z~/.ibus/tablesz	debug.logzsetup-debug.logT)�exist_okz(The user database %s does not exist yet.)�id�tabkeys�phrase�freq�	user_freq�versionz.The user database %s seems to be incompatible.z0There is no version information in the database.�0.0)�old_database_versionz�The version of the database does not match (too old or too new?). ibus-table wants version=%s But the  database actually has version=%sz�The number of columns of the database does not match. ibus-table expects %s columns. But the database actually has %s columns. But the versions of the databases are identical. This should never happen!z-%Y-%m-%d_%H:%M:%Sz+Renaming the incompatible database to "%s".z-shmz-walz$Creating a new, empty database "%s".z�If user phrases were successfully recovered from the old, incompatible database, they will be used to initialize the new database.zCompatible database %s found.z-Unexpected error trying to find user databasezConnect to the database %s.a�
                ATTACH DATABASE "%s" AS user_db;
                PRAGMA user_db.encoding = "UTF-8";
                PRAGMA user_db.case_sensitive_like = true;
                PRAGMA user_db.page_size = 4096;
                PRAGMA user_db.cache_size = 20000;
                PRAGMA user_db.temp_store = MEMORY;
                PRAGMA user_db.journal_mode = WAL;
                PRAGMA user_db.journal_size_limit = 1000000;
                PRAGMA user_db.synchronous = NORMAL;
            zCould not open the database %s.r+r���rcrdrerfz�
            INSERT INTO user_db.phrases (tabkeys, phrase, freq, user_freq)
            VALUES (:tabkeys, :phrase, :freq, :user_freq)
            zError inserting old phrases�PRAGMA wal_checkpoint;)M�intr(�os�getenv�DEBUG_LEVEL�	TypeError�
ValueError�old_phrasesr*�_user_db�reset_phrases_cache�path�isfiler%�connectr�printrrr�
executescriptr2�uuid4�time�strftime�_default_ime_attributes�sortedrr
�ime_propertiesr�_mlen�_snum�
is_chinese�_is_chinese�is_cjk�_is_cjk�lowerrH�	get_rulesrS�get_possible_tabkeys_lengths�possible_tabkeys_lengths�get_start_chars�
startchars�join�ibus_table_location�	data_home�basename�replace�
cache_path�load_phrases_cache�isdir�
expanduser�access�F_OK�unlink�shutil�copytree�rmtree�symlink�makedirs�exists�debug�get_database_desc�DATABASE_VERSION�%get_number_of_columns_of_phrase_table�len�extract_user_phrases�rename�init_user_db�
create_tables�append�executemany�commit�create_indexes�generate_userdb_desc)rr*r+r,r-�
select_sqlstr�
insert_sqlstrr[�sqlargs�tables_path�
cache_name�old_tables_path�desc�phrase_table_column_names�	timestamp�sqlargs_old_phrasesrdrrrrr�s����������	�
���
����������������� �!�"�#�$�%
�+���

��




��
��
�
�

���
�
��
�����
��
�����

�
�

�

��

zTabSqliteDb.__init__rTrcrdrf�databaser�cCs~tdkr
t�d||||�|r|sdSd|}|||d�}z|j�||�|r,|j��|�|�WdSt�d�YdS)zupdate phrase freqsrz-tabkeys=%s phrase=%s user_freq=%s database=%sNzu
        UPDATE %s.phrases SET user_freq = :user_freq
        WHERE tabkeys = :tabkeys AND phrase = :phrase
        ;)rfrcrdz,Unexpected error updating phrase in user_db.)rqrr�rrr��invalidate_phrases_cacher)rrcrdrfr�r�rr�rrr�
update_phrase�s*���
zTabSqliteDb.update_phrasecCs0|��|jdurdS|j��|j�d�dS)z1
        Trigger a checkpoint operation.
        Nrm)�save_phrases_cacherurr�rrrrr�
sync_usrdb�s


zTabSqliteDb.sync_usrdbcCstdkr	t�d�i|_dS)z.
        Make the phrases cache empty
        rzreset_phrases_cache()N)rqrr��_phrases_cacherrrrrv�s

zTabSqliteDb.reset_phrases_cachecCsTtdkr	t�d�td|jd�D]}|j�|d|��r'|j�|d|��qdS)u�
        Delete all phrases starting with “tabkeys” from
        the phrases cache.

        :param tabkeys: The keys typed
        rzinvalidate_phrases_cache()rN)rqrr��ranger�r�r�pop)rrc�irrrr��s
��z$TabSqliteDb.invalidate_phrases_cachecCs�tdkr	t�d�z t�t|jd��|_|j�d�}|r!||j	kr'i|_WdSWdSt
yAtdkr>t�d|j�YdSYdStyXtdkrUt�d|j�YdSYdSt�d|j�YdS)	z.
        Load phrases cache from disk
        rzload_phrases_cache()�rr3zFile %s not foundzPermission error reading %szUnknown error reading %sN)rqrr��json�load�openr�r�rr��FileNotFoundError�PermissionError)r�snumrrrr��s,
�
��
��zTabSqliteDb.load_phrases_cachecCshtdkr	t�d�z|j|jd<|jd}t�|jt|d��t	�
||j�WdSt�d�YdS)z.
        Save phrases cache from disk
        rzsave_phrases_cache()r3z.tmp�wz)Unexpected error in save_phrases_cache().N)rqrr�r�r�r�r��dumpr�ror�r)r�_cache_pathrrrr�s

zTabSqliteDb.save_phrases_cachecCs@|j�d�}|r|�d�}|D]}|���d�dkrdSqdS)zg
        Check whether this input method is classified as Chinese
        in the the database.
        r7�,�zh���TF)r�r�splitr��find)rr7�langs�langrrrr�s
�zTabSqliteDb.is_chinesecCsH|j�d�}|r"|�d�}|D]}dD]
}|���|�r dSqqdS)zx
        Check whether this input method is classified as Chinese,
        Japanese, or Korean in the database.
        r7r�)r��ja�koTF)r�rr��strip�
startswith)r�
languages_strr7�languager�rrrr�s
��zTabSqliteDb.is_cjkcCs$|j�d�}|dvrt|d�SdS)a�
        Get the default Chinese mode from the database

        0 means to show simplified Chinese only
        1 means to show traditional Chinese only
        2 means to show all characters but show simplified Chinese first
        3 means to show all characters but show traditional Chinese first
        4 means to show all characters

        If no mode is specified in the database, return 4 to avoid all
        filtering of characters.
        r8)�cm0�cm1�cm2�cm3�cm4r��)r�rrn)rr8rrr�get_chinese_mode+s
zTabSqliteDb.get_chinese_modecCs|j�d�}|r
|SdS)zK
        Get the keys used to select a candidate from the database
        r?r@�r�r)r�retrrr�get_select_keys=szTabSqliteDb.get_select_keysc	Cs,z	t|j�d��WSttfyYdSw)zAGet the default orientation of the lookup table from the databaserWr)rnr�rrrrsrrrr�get_orientationFs
�zTabSqliteDb.get_orientationcCsf|dkr"d|}|j�|�d|}|j�|�d|}|j�|�d|}|j�|�|j��dS)z%Create tables that contain all phrase�mainzp
            CREATE TABLE IF NOT EXISTS %s.goucima
            (zi TEXT PRIMARY KEY, goucima TEXT);
            zp
            CREATE TABLE IF NOT EXISTS %s.pinyin
            (pinyin TEXT, zi TEXT, freq INTEGER);
            zk
            CREATE TABLE IF NOT EXISTS %s.suggestion
            (phrase TEXT, freq INTEGER);
            z�
        CREATE TABLE IF NOT EXISTS %s.phrases
        (id INTEGER PRIMARY KEY, tabkeys TEXT, phrase TEXT,
        freq INTEGER, user_freq INTEGER);
        N)rrr�)rr�rrrrr�Ms$����zTabSqliteDb.create_tables�attrscCs�d}d}d}|D]!\}}||d�}|j�||���r"|j�||�q|j�||�q|j��t|j|jd�|_t|j�d��|_	|�
�|_|j�d���dk|_
|��|_d	S)
z�Update or insert attributes in ime table, attrs is a iterable object
        Like [(attr,val), (attr,val), ...]

        This is called only by tabcreatedb.py.
        z+SELECT val from main.ime WHERE attr = :attrz2UPDATE main.ime SET val = :val WHERE attr = :attr;z6INSERT INTO main.ime (attr, val) VALUES (:attr, :val);rZr]r;rHrEN)rrrr�r
rr�rnrr�r�r�r�rHr�rS)rr�r��
update_sqlstrr�r[r\r�rrr�
update_imehs,�

�
��zTabSqliteDb.update_imec	Cs2i}t�d�}t�d�}|jsiSzx|j�d�}g}|r#|���d�}|D]a}|�|�}|r�g}|�d�dkr@t	|�d��|d<|�d	��d
�}	t
|	�|jkrYtd|�W|S|	D]}
|�|
�}|ru|�
t	|�d��t	|�d��f�q[||t	|�d��<q%td|�q%W|Sty�t�d
�Y|Sw)a�Get phrase construct rules

        Example:

        The wubi-jidian86.txt table source contains:

        RULES = ce2:p11+p12+p21+p22;ce3:p11+p21+p31+p32;ca4:p11+p21+p31+p-11

        and the return value of this function becomes:

        {2: [(1, 1), (1, 2), (2, 1), (2, 2)],
         3: [(1, 1), (2, 1), (3, 1), (3, 2)],
         'above': 4,
         4: [(1, 1), (2, 1), (3, 1), (-1, 1)]}
        zc([ea])(\d):(.*)zp(-{0,1}\d)(-{0,1}\d)rS�;r�arj�aboverk�+zrule: "%s" over max key lengthznot a legal rule: "%s"z Unexpected error in get_rules().)�re�compilerHr�rr�r��match�grouprnr�r�rzr��	Exceptionrr)rrS�patt_r�patt_p�
_rules_str�_rules�rule�res�cms�_cms�_cm�cm_resrrrr��sH


�
�����zTabSqliteDb.get_rulesc	s��jr�jd}�fdd�td|d�D�dd�Sz
t�j�d��}Wn
ttfy1d}Ynw|dkr@tt|�jd��SgS)	u�Return a list of the possible lengths for tabkeys in this table.

        Example:

        If the table source has rules like:

            RULES = ce2:p11+p12+p21+p22;ce3:p11+p21+p22+p31;ca4:p11+p21+p31+p41

        self._rules will be set to

            self._rules = {
                2: [(1, 1), (1, 2), (2, 1), (2, 2)],
                3: [(1, 1), (1, 2), (2, 1), (3, 1)],
                4: [(1, 1), (2, 1), (3, 1), (-1, 1)],
                'above': 4}

        and then this function returns “[4, 4, 4]”

        Or, if the table source has no RULES but LEAST_COMMIT_LENGTH=2
        and MAX_KEY_LENGTH = 4, then it returns “[2, 3, 4]”

        I cannot find any tables which use LEAST_COMMIT_LENGTH though.
        r�csg|]	}t�j|��qSr)r�rS��.0�xrrr�
<listcomp>�sz<TabSqliteDb.get_possible_tabkeys_lengths.<locals>.<listcomp>rjrNrTr)	rSr�rnr�rrrrs�listr�)r�max_len�least_commit_lenrrrr��s
$
��z(TabSqliteDb.get_possible_tabkeys_lengthscCs|j�d�S)z"return possible start chars of IMErVr�rrrrr��szTabSqliteDb.get_start_charscCs|j�d�}|S)z6Get the characters which engine should not change freq�no_check_charsr�)r�_charsrrr�get_no_check_chars�szTabSqliteDb.get_no_check_charsr��phrasesc	Cs�tdkrt�dtt|���dd|i}g}|D]\}}}}|�||||d��|�|�q|j�||�|j�	�|j�
d�dS)u�Add many phrases to database fast. Used by tabcreatedb.py when
        creating the system database from scratch.

        “phrases” is a iterable object which looks like:

            [(tabkeys, phrase, freq ,user_freq),
             (tabkeys, phrase, freq, user_freq), ...]

        This function does not check whether phrases are already
        there.  As this function is only used while creating the
        system database, it is not really necessary to check whether
        phrases are already there because the database is initially
        empty anyway. And the caller should take care that the
        “phrases” argument does not contain duplicates.

        rzlen(phrases)=%s��
        INSERT INTO %(database)s.phrases
        (tabkeys, phrase, freq, user_freq)
        VALUES (:tabkeys, :phrase, :freq, :user_freq);
        r�rlrmN)rqrr�r�r�r�r�rr�r�r)	rrr�r��insert_sqlargsrcrdrerfrrr�add_phrases�s"��
zTabSqliteDb.add_phrasesrecCs�tdkr
t�d||||�|r|sdSdd|i}||d�}|j�||���}	|	r7tdkr5t�d|||	�dSdd|i}
||||d	�}tdkrOt�d
|
|�z|j�|
|�|r^|j��|�|�WdSt�d�YdS)zaAdd phrase to database, phrase is a object of
        (tabkeys, phrase, freq ,user_freq)
        rz4add_phrase tabkeys=%s phrase=%s freq=%s user_freq=%sNzk
        SELECT * FROM %(database)s.phrases
        WHERE tabkeys = :tabkeys AND phrase = :phrase;
        r��rcrdzKselect_sqlstr=%(sql)s select_sqlargs=%(arg)s already there!: results=%(r)s rrlz"insert_sqlstr=%s insert_sqlargs=%sz Unexpected error in add_phrase())	rqrr�rrrr�r�r)rrcrdrerfr�r�r��select_sqlargsrr�rrrr�
add_phrasesN��
����
zTabSqliteDb.add_phrase�goucimascCsrd}g}|D]\}}|�||d��qz|j��|j�||�|j��|j�d�WdSt�d�YdS)zyAdd goucima into database, goucimas is iterable object
        Like goucimas = [(zi,goucima), (zi,goucima), ...]
        zP
        INSERT INTO main.goucima (zi, goucima) VALUES (:zi, :goucima);
        )�zi�goucimarmz"Unexpected error in add_goucima().N)r�rr�r�rrr)rrrr�rr
rrr�add_goucima@s

zTabSqliteDb.add_goucima�pinyinscCs�d|}d}|D]>\}}}|d7}|�dd��dd��dd	��d
d��dd
�}z
|j�||||d��WqtyFt�d||||�Yqw|j��dS)zxAdd pinyin to database, pinyins is a iterable object
        Like: [(zi,pinyin, freq), (zi, pinyin, freq), ...]
        zX
        INSERT INTO %s.pinyin (pinyin, zi, freq) VALUES (:pinyin, :zi, :freq);
        rr�1�!�2�@�3�#r<�$�5�%)�pinyinrrezHError when inserting into pinyin table. count=%s pinyin=%s zi=%s freq=%sN)r�rrr�rrr�)rrr�r�countrrrerrr�
add_pinyinRsB����������zTabSqliteDb.add_pinyin�suggestionsc
Csjd|}d}|D]%\}}|d7}z|j�|||d��Wqty-t�d|||�Yqw|j��dS)z�Add suggestion phrase to database, suggestions is a iterable object
        Like: [(phrase, freq), (phrase, freq), ...]
        zS
        INSERT INTO %s.suggestion (phrase, freq) VALUES (:phrase, :freq);
        rr)rdrezFError when inserting into suggestion table. count=%s phrase=%s freq=%sN)rrr�rrr�)rrr�rrrdrerrr�add_suggestionos"�
���zTabSqliteDb.add_suggestioncCs*d}|j�|�|j�d�|j��dS)ze
        Optimize the database by copying the contents
        to temporary tables and back.
        a�
            CREATE TABLE tmp AS SELECT * FROM main.phrases;
            DELETE FROM main.phrases;
            INSERT INTO main.phrases SELECT * FROM tmp ORDER BY
            tabkeys ASC, phrase ASC, user_freq DESC, freq DESC, id ASC;
            DROP TABLE tmp;
            CREATE TABLE tmp AS SELECT * FROM main.goucima;
            DELETE FROM main.goucima;
            INSERT INTO main.goucima SELECT * FROM tmp ORDER BY zi, goucima;
            DROP TABLE tmp;
            CREATE TABLE tmp AS SELECT * FROM main.pinyin;
            DELETE FROM main.pinyin;
            INSERT INTO main.pinyin SELECT * FROM tmp ORDER BY pinyin ASC, freq DESC;
            DROP TABLE tmp;
            CREATE TABLE tmp as SELECT * FROM main.suggestion;
            DELETE FROM main.suggestion;
            INSERT INTO main.suggestion SELECT * FROM tmp ORDER by phrase ASC, freq DESC;
            DROP TABLE tmp;
            zVACUUM;N)rr{r�)rrrrr�optimize_database�szTabSqliteDb.optimize_database�	_databasecC�tdkrt�d�dSdS)z�Drop the indexes in the database to reduce its size

        We do not use any indexes at the moment, therefore this
        function does nothing.
        rzdrop_indexes()N�rqrr�)rrrrr�drop_indexes�s�zTabSqliteDb.drop_indexes�_commitcCr )u�Create indexes for the database.

        We do not use any indexes at the moment, therefore
        this function does nothing. We used indexes before,
        but benchmarking showed that none of them was really
        speeding anything up, therefore we deleted all of them
        to get much smaller databases (about half the size).

        If some index turns out to be very useful in future, it could
        be created here (and dropped in “drop_indexes()”).
        rzcreate_indexes()Nr!)rrr#rrrr��s�zTabSqliteDb.create_indexescCs*z|�d�}W|Styd}Y|Sw)z�
        Encode a string in Big5 or, if that is not possible,
        return something higher than any Big5 code.

        :param phrase: String to be encoded in Big5 encoding
        �Big5���)�encode�UnicodeEncodeError)rrd�big5rrr�	big5_code�s��zTabSqliteDb.big5_coderr��
typed_tabkeys�
candidates�chinese_modecs�d}tj�|j��dd�}|dvr|j�ndd��|jr#�fdd��ndd��|d	vrH|jrH|d
kr5d�nd
�t|����fdd�d
�d|�St|���fdd�d
�d|�S)u 
        “candidates” is an array containing something like:
        [(tabkeys, phrase, freq, user_freq), ...]

        “typed_tabkeys” is key sequence the user really typed, which
        maybe only the beginning part of the “tabkeys” in a matched
        candidate.
        �dr_r��cangjie3�cangjie5zcangjie-bigz
quick-classic�quick3�quick5cS�dS�Nr%r�r�rrr�<lambda>��z-TabSqliteDb.best_candidates.<locals>.<lambda>cs"t�|dd�ko|ddv�S)Nr�z!@#$%)rnr5)r*rrr6�s"cSr3)Nrrr5rrrr6�r7)rjrkrjrcspt�|dk��|d�d|d�t�|d�@d|dt|d�|d�|dd�t|dd�f	S)Nrr�rkrrj)rn�chinese_variants�detect_chinese_categoryr��ordr5��bitmask�code_point_function�pinyin_exact_match_functionr*rrr6�s 
�

��

�rNc
s^t�|dk��|d�d|dd|dt|d�|d�|dd�t|dd�fS)Nrr�rkrjr�rnr�r:r5)r=r>r*rrr6�s
�



�)rorwr�r*r�r)r�r�)rr*r+r,�maximum_number_of_candidates�engine_namerr;r�best_candidates�s,
���
�zTabSqliteDb.best_candidates�onechar�single_wildcard_char�multi_wildcard_charrNrKc
Cs|sgS|j�|�}|r|Sd}	|rd}	|js|r dd|	i}
ndd|	i}
d}dD]
}|||fvr4|}q*|}
|
�|||�}
d||fvrM|
�d|d�}
d	||fvr[|
�d	|d	�}
|rc|
�|d	�}
|rk|
�|d
�}
|rq|
d
7}
|
|d�}tdkr�t�d
|
t|��|j�	|
|��
�}d}|dkr�d}n|dkr�d}|s�|}ng}|D]}|t�|d�@r�|�
|�q�i}|D]1}|d|df}||vr�|||<q�|�||t|d||d�t|d||d�ffg�q�|j||��|d�}tdk�rt�dt|��||j|<|S)zE
        Get matching phrases for tabkeys from the database.
        rz AND length(phrase)=1 a�
            SELECT tabkeys, phrase, freq, user_freq FROM
            (
                SELECT tabkeys, phrase, freq, user_freq FROM main.phrases
                WHERE tabkeys LIKE :tabkeys ESCAPE :escapechar %(one_char_condition)s
                UNION ALL
                SELECT tabkeys, phrase, freq, user_freq FROM user_db.phrases
                WHERE tabkeys LIKE :tabkeys ESCAPE :escapechar %(one_char_condition)s
            )
            �one_char_conditionz�
            SELECT tabkeys, phrase, freq, user_freq FROM main.phrases
            WHERE tabkeys LIKE :tabkeys ESCAPE :escapechar %(one_char_condition)s
            u☺z!@#r�_�%%)rc�
escapecharrzsqlstr=%s sqlargs=%sNrrjrk�r*r+r,zbest=%s)r�rrHr�rqrr�rrrrr8r9r��update�maxrC�values)rrcrDr,rErFrNrK�bestrGrrJ�char�tabkeys_for_liker��unfiltered_resultsr<rr�phrase_frequenciesrrrr�select_words	s�
	������
�
�
��
��

zTabSqliteDb.select_wordscCs�|sgSd}|}|r|�|d�}|r|�|d�}|d7}d|i}|j�||���}d}	|dkr2d}	n|dkr8d}	g}
|D]!\}}}
|	sM|
�|||
df�q<|	t�|�@r]|
�|||
df�q<|j||
|d	�S)
zh
        Get Chinese characters matching the pinyin given by tabkeys
        from the database.
        z}
        SELECT pinyin, zi, freq FROM main.pinyin WHERE pinyin LIKE :tabkeys
        ORDER BY freq DESC, pinyin ASC;
        rHrIrcNrrrjrK)r�rrrr�r8r9rC)rrcr,rErFrrQr�rr<rSrrrerrr�#select_chinese_characters_by_pinyinps@
����z/TabSqliteDb.select_chinese_characters_by_pinyin�prefixc	s�|sgSd}|d}d|i}|j�||���}i}|D] \}}||vr*||f||<q|�||t|||d�ffg�q|��}	tdkrLt�dt	|	��d}
t
j�|j
��dd�}|d	vra|j�nd
d��t|	�fdd�d
�d|
�S)zK
        Get Chinese phrase matching the prefix from the database.
        z�
        SELECT phrase, freq FROM main.suggestion WHERE phrase LIKE :prefix
        ORDER BY length(phrase) DESC, freq DESC, phrase ASC;
        rIrVrz
candidates=%sr-r_rr.cSr3r4rr5rrrr6�r7z9TabSqliteDb.select_suggestion_candidate.<locals>.<lambda>csVtt|d��d|d�|dd��|dd�t|dd�t|dd�fS)Nrr�rr@r5�r=rrr6�s
�r?N)rrrrLrMrNrqrr�rrorwr�r*r�r)r�)rrVr�prefix_for_liker�rrSrdrer+rArBrrWr�select_suggestion_candidate�s8��
�
�z'TabSqliteDb.select_suggestion_candidatecCs`z$d}|j�|�d}|j�|dtf�d}|j�|d�|j��WdSt�d�YdS)z{
        Add a description table to the user database

        This adds the database version and  the create time
        zBCREATE TABLE IF NOT EXISTS user_db.desc (name PRIMARY KEY, value);z2INSERT OR IGNORE INTO user_db.desc  VALUES (?, ?);rgzMINSERT OR IGNORE INTO user_db.desc  VALUES (?, DATETIME("now", "localtime"));)zcreate-timez+Unexpected error in generate_userdb_desc().N)rr{rr�r�rr)r�	sqlstringrrrr��s��z TabSqliteDb.generate_userdb_desc�db_filecCs:|dkrdSt�|�st�|�}|�d�|��dSdS)z�
        Initialize the user database unless it is an in-memory database

        :param db_file: Full path of the database file.
        :type db_file: String
        r`Nax
                PRAGMA encoding = "UTF-8";
                PRAGMA case_sensitive_like = true;
                PRAGMA page_size = 4096;
                PRAGMA cache_size = 20000;
                PRAGMA temp_store = MEMORY;
                PRAGMA journal_mode = WAL;
                PRAGMA journal_size_limit = 1000000;
                PRAGMA synchronous = NORMAL;
            )rwr�r%ryr{r�)rr[rrrrr��s



�zTabSqliteDb.init_user_dbcCs\t�|�sdSz t�|�}i}|�d���D]
}|d||d<q|��|WSYdS)z�
        Get the description table from the database

        :param db_file: Full path of the database file.
        :type db_file: String
        :rtype: Dictionary
        NzSELECT * FROM desc;rr)rwr�r%ryrr�close)rr[rr��rowrrrr��s

zTabSqliteDb.get_database_desccCszt�|�sdSz/t�|�}|�d���}d�|dd���}t�	d|�}|r4|�
d��d�}t|�WSWdSYdS)a1
        Get the number of columns in the 'phrases' table in
        the database in db_file.

        Determines the number of columns by parsing this:

        sqlite> select sql from sqlite_master where name='phrases';
        CREATE TABLE phrases
                (id INTEGER PRIMARY KEY, tabkeys TEXT, phrase TEXT,
                freq INTEGER, user_freq INTEGER)
        sqlite>

        This result could be on a single line, as above, or on multiple
        lines.

        :param db_file: Full path of the database file.
        :rtype: Integer
        rz3select sql from sqlite_master where name='phrases';� z
.*\((.*)\)rr�)
rwr�r%ryrrr��
splitlinesr�r�r�r�r�)rr[r�tp_res�stringr��tprrrr�s"

��
z1TabSqliteDb.get_number_of_columns_of_phrase_tablercCsN|sdSd}|j�|d|i���}d}|r|dd}tdkr%t�d|�|S)zGet goucima of given characterrz0SELECT goucima FROM main.goucima WHERE zi = :zi;rrrz
goucima=%s)rrrrqrr�)rrrrr
rrr�get_goucima3szTabSqliteDb.get_goucimacCsNtdkrt�d||j�|sdSt|�dkr|�|�S|js dSt|�|jvr/|jt|�}n$t|jdt�rIt|�|jdkrI|j|jd}n
t�d||j�dSt|t�sit|�|jkrit�d||j�dSd}t|t�rrdS|D]&\}}|dkr�|d8}|dkr�|d8}|�||�|}|s�dS||7}qttdkr�t�d|�|S)	uRParse phrase to get its table code

        Example:

        Let’s assume we use wubi-jidian86. The rules in the source of
        that table are:

          RULES = ce2:p11+p12+p21+p22;ce3:p11+p21+p31+p32;ca4:p11+p21+p31+p-11

        “ce2” is a rule for phrases of length 2, “ce3” is a rule
        for phrases of length 3, “ca4” is a rule for phrases of
        length 4 *and* for all phrases with a length greater then
        4. “pnm” in such a rule means to use the n-th character of
        the phrase and take the m-th character of the table code of
        that character. I.e. “p-11” is the first character of the
        table code of the last character in the phrase.

        Let’s assume the phrase is “天下大事”. The goucima (構詞碼
        = “word formation keys”) for these 4 characters when
        using the wubi-jidian86 table are:

            character goucima
            天        gdi
            下        ghi
            大        dddd
            事        gkvh

        (If no special goucima are defined by the user, the longest
        encoding for a single character in a table is the goucima for
        that character).

        The length of the phrase “天下大事” is 4 characters,
        therefore the rule ca4:p11+p21+p31+p-11 applies, i.e. the
        table code for “天下大事” is calculated by using the first,
        second, third and last character of the phrase and taking the
        first character of the goucima for each of these. Therefore,
        the table code for “天下大事” is “ggdg”.

        rzphrase=%s rules%srr�z2No rule for this phrase length. phrase=%s rules=%sz6Rule exceeds maximum key length. rule=%s self._mlen=%srz
tabkeys=%s)	rqrr�rSr�rc�
isinstancernr�)rrdr�rcr�ma�tabkeyrrr�parse_phrase@sN(
��

zTabSqliteDb.parse_phrasecCsftdkrt�d||�|r|sdSd}|d|d�}|j�||���}tdkr/t�d|||�t|�S)u�
        Checks whether “phrase” can be matched in the system database
        with a key sequence *starting* with “tabkeys”.
        r�tabkeys=%s phrase=%sFzf
        SELECT * FROM main.phrases
        WHERE tabkeys LIKE :tabkeys AND phrase = :phrase;
        rIrztabkeys=%s phrase=%s results=%s)rqrr�rrr�bool)rrcrdrr�rrrr�is_in_system_database�s�z!TabSqliteDb.is_in_system_databasecCsntdkrt�d||�|r|sdSd}||d�}|j�||���}tdkr+t�d|�|r5t|dd�SdS)u
        Return how often a conversion result “phrase” for the typed keys
        “tabkeys” has been happened by checking the user database.

        :param tabkeys: The keys typed
        :param phrase: A conversion result for these tabkeys
        rrhrz�
        SELECT sum(user_freq) FROM user_db.phrases
        WHERE tabkeys = :tabkeys AND phrase = :phrase GROUP BY tabkeys, phrase;
        rz	result=%s)rqrr�rrrrn)rrcrdrr�rrrr�user_frequency�s
zTabSqliteDb.user_frequencycCsVtdkrt�d||�|r|sdS|jr|tvrdS|sM|jr"|js$dS|�|�}|s-dS|j||d�r6dS|j	||d�dkrAdS|j
||dddd�dS|j||d�rw|j	||d�}|dkrk|j|||dd	�dS|j
||dddd�dS|jr}|jsdS|�|�}|s�dS|j	||d�}|dkr�|j|||dd	�dS|j
||dddd�dS)
aAdjust user_freq in user database if necessary.

        Also, if the phrase is not in the system database, and it is a
        Chinese table, and defining user phrases is allowed, add it as
        a user defined phrase to the user database if it is not yet
        there.
        rztabkey=%s phrase=%sNrrr�r+)rcrdrerfr�)rcrdrf)rqrr�r��CHINESE_NOCHECK_CHARSrHr�rgrjrkr
r�)rrcrdrKrfrrr�check_phrase�sX

�

�
�


�
�zTabSqliteDb.check_phrasecCs0d}d|i}|j�||���}dd�|D�}|S)uu
        Return the list of possible tabkeys for a phrase.

        For example, if “phrase” is “你” and the table is wubi-jidian.86.txt,
        the result will be ['wq', 'wqi', 'wqiy'] because that table
        contains the following 3 lines matching that phrase exactly:

        wq	你	597727619
        wqi	你	1490000000
        wqiy	你	1490000000
        zo
        SELECT tabkeys FROM main.phrases WHERE phrase = :phrase
        ORDER by length(tabkeys) ASC;
        rdcSsg|]}|d�qS)rrr�rrrr�sz,TabSqliteDb.find_zi_code.<locals>.<listcomp>)rrr)rrdrr�r�list_of_possible_tabkeysrrr�find_zi_codes
zTabSqliteDb.find_zi_codecCslt�d||||�|s
dS|rdd|i}ndd|i}||d�}|j�||�|r/|j��|�|�dS)z$Remove phrase from database
        z5Removing tabkeys=%s, phrase=%s, database=%s commit=%sNzu
            DELETE FROM %(database)s.phrases
            WHERE tabkeys = :tabkeys AND phrase = :phrase;
            r�z^
            DELETE FROM %(database)s.phrases
            WHERE phrase = :phrase;
            r)r�inforrr�r�)rrcrdr�r��
delete_sqlstr�delete_sqlargsrrr�
remove_phrases"���

zTabSqliteDb.remove_phrasecCsZt�d�z|j�d�|j��|j�d�|��WdSty,t�d�YdSw)zv
        Remove all phrases from the user database, i.e. delete all the
        data learned from user input.
        z,Removing all phrases from the user database.zDELETE FROM user_db.phrases;rmz4Unexpected error removing all phrases from database.N)rrprrr�rvr�rrrrr�remove_all_phrases_from_user_db2s


��z+TabSqliteDb.remove_all_phrases_from_user_dbrh�
database_filericCsLt�d�z�t�|�}|�d�|dkr6|�d���}|��t|dd�d�}t�dt|��|d	d	�WSg}|�d
���}|D]=}d}d|d
i}|j	�||���}	|	rh|�
|	d
d
|d
d
|df�qA|�|d
�}
|
r~|�
|
|d
d|df�qA|��t|dd�d�}t�dt|��|d	d	�WSt�d�gYS)z"extract user phrases from databasezBTrying to recover the phrases from the old, incompatible database.rmr	z�
                    SELECT tabkeys, phrase, freq, sum(user_freq) FROM phrases
                    GROUP BY tabkeys, phrase, freq;
                    cS�|d|d|d|dfS�Nrrrjrkrr5rrrr6V�z2TabSqliteDb.extract_user_phrases.<locals>.<lambda>r?z3Recovered phrases from the old database: phrases=%sNz;SELECT phrase, sum(user_freq) FROM phrases GROUP BY phrase;z�
                SELECT tabkeys FROM main.phrases WHERE phrase = :phrase
                ORDER BY length(tabkeys) DESC;
                rdrrr�cSrvrwrr5rrrr6|rxz8Recovered phrases from the very old database: phrases=%sz*Unexpected error in extract_user_phrases())
rr�r%ryrrr\r�rrr�rgr)rrurirrrrrr��tabkeys_resultsrcrrrr�Asn�

������������
z TabSqliteDb.extract_user_phrases)rrFF)rrrr+T)r
N)r)r�)rrrrr�T)T)rrr�)rFr�rrFF)rr�rr)rr)rrF)rrr+T)rrh):r!r"r#r$r(rirrnr�r�rvr�r�r�r�r�r�r�r�r�rrr�rrrr�r�r�rrr
rrrrr"r��bytesr)rCrTrUrYr�r�rr�r�rcrgrjrkrmrorsrtr�rrrrr)ys������
�������
�





	20$���
�)�������
�8���
� ���
�
	����
�D��������
�i�����
�.��
�
0%
U���
�����
�?�����
�
����r))r$�typingrrrrrrrro�os.pathrwr�r%r2r}r��loggingr�r8r��	getLoggerrrnrqr�rlr
r)rrrr�<module>s2
 &

Youez - 2016 - github.com/yon3zu
LinuXploit