Study note

学习笔记：分布式训练模型

2025-04-16 没有评论

分布式训练模型（Distributed Training）分布式训练是一种将深度学习模型的训练过程分布在多个设备（如GPU）上的技术，用于提升训练速度、处理更大的模型和数据集。 🔹 分布式训练的四种主要方式 1. Data Parallelism（DP） – 数据并行原理：将数据切分 […]

Study note

学习笔记：AI模型能力评估

2025-04-08 没有评论

AI能力评估方法调研报告引言：AI能力评估的关键作用随着人工智能模型能力的不断增强及其在各个领域的广泛应用，对这些模型进行稳健且可靠的评估变得至关重要。评估在通用人工智能开发过程的每个阶段都至关重要，包括模型选择、提示工程和模型定制。理解不同AI模型的优势和劣势对于在各种应用中做出明智的决策至关 […]

Study note

MCP简介

2025-04-07 没有评论

PDF文件（高清）：MCP简介_update_20250417_pdf XMind文件（源文件）：MCP简介_update_20250417_xmind PDF文件（高清）：MCP简介 XMind文件（源文件）：MCP简介.xmind

Study note

简单谈谈Amazon Aurora DSQL

2024-12-10 没有评论

PDF文件（高清）： Amazon Aurora DSQL XMind文件（源文件）： Amazon Aurora DSQL （思维导图太大了，blog放不下，各位可以选取PDF来看，比较清晰。）参考：【AWS re:Invent 2024】 AWS re:Invent 2024 &#821 […]

Study note

【译文】在Amazon Aurora DSQL中的并发控制

2024-12-05 没有评论

在任何数据库管理系统（DBMS）中，并发控制在确保多个事务能够同时进行而不会导致数据不一致或损坏方面起着至关重要的作用。它指的是管理事务如何交互的机制，特别是在它们访问共享资源时。分布式数据库管理系统（D-DBMS）由于数据分布在多个节点上而引入了复杂性，导致了数据复制、网络延迟和保持一致性等挑战， […]

Study note

【译文】介绍Amazon Aurora DSQL

2024-12-05 没有评论

关系数据库是各种应用程序和服务的强大且灵活的构建模块——从微服务和网站，到移动后端和SaaS应用程序。十年前，我们推出了Amazon Aurora，提供了无与伦比的高性能和高可用性，并完全兼容MySQL和PostgreSQL，成本仅为商业数据库的十分之一。我们听到客户希望有一种关系数据库，更易于管理 […]

Study note

读书笔记-《Best Practices for Gathering Optimizer Statistics with Oracle Database 19c》

2024-05-14 没有评论

调整SET_TABLE_PREFS，按照表来单独设置，减低stale的百分比，比如变化1%就收集统计信息调整 ESTIMATE_PERCENT的比例，不用AUTO_SAMPLE_SIZE，而用更低的采样比例 High-frequency Optimizer Statistics Collectio […]

Study note

数据库内查询pg的表结构定义

2020-09-24 没有评论

需要利用到plperlu和自己写一个system函数。

-bash-4.2$ psql
psql (9.6.2)
Type "help" for help.

postgres=# create extension plperlu;
CREATE EXTENSION
postgres=# \dx
                      List of installed extensions
  Name   | Version |   Schema   |              Description               
---------+---------+------------+----------------------------------------
 plperlu | 1.0     | pg_catalog | PL/PerlU untrusted procedural language
 plpgsql | 1.0     | pg_catalog | PL/pgSQL procedural language
(2 rows)

1

2

3

4

5

6

7

8

9

10

11

12

13

-bash-4.2$ psql

psql (9.6.2)

Type "help" for help.

postgres=# create extension plperlu;

CREATE EXTENSION

postgres=# \dx

List of installed extensions

Name | Version | Schema | Description

---------+---------+------------+----------------------------------------

plperlu | 1.0 | pg_catalog | PL/PerlU untrusted procedural language

plpgsql | 1.0 | pg_catalog | PL/pgSQL procedural language

(2 rows)

postgres=# CREATE OR REPLACE FUNCTION system(text) RETURNS text
postgres-# AS 'my $cmd=shift; return `cd /tmp;$cmd`;' LANGUAGE plperlu;
CREATE FUNCTION
postgres=# 
postgres=#

1

2

3

4

5

postgres=# CREATE OR REPLACE FUNCTION system(text) RETURNS text

postgres-# AS 'my $cmd=shift; return `cd /tmp;$cmd`;' LANGUAGE plperlu;

CREATE FUNCTION

postgres=#

postgres=# select system('pg_dump -s -t orasup_test1 dbinfo2 |egrep -v "^--|^$"');
                       system                        
-----------------------------------------------------
 SET statement_timeout = 0;                         +
 SET lock_timeout = 0;                              +
 SET idle_in_transaction_session_timeout = 0;       +
 SET client_encoding = 'UTF8';                      +
 SET standard_conforming_strings = on;              +
 SET check_function_bodies = false;                 +
 SET client_min_messages = warning;                 +
 SET row_security = off;                            +
 SET search_path = public, pg_catalog;              +
 SET default_tablespace = '';                       +
 SET default_with_oids = false;                     +
 CREATE TABLE orasup_test1 (                        +
     a integer,                                     +
     b character varying(200)                       +
 );                                                 +
 ALTER TABLE orasup_test1 OWNER TO djidba_rw;       +
 CREATE INDEX idx_b ON orasup_test1 USING btree (b);+
 
(1 row)

postgres=# 
postgres=#

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

postgres=# select system('pg_dump -s -t orasup_test1 dbinfo2 |egrep -v "^--|^$"');

system

-----------------------------------------------------

SET statement_timeout = 0; +

SET lock_timeout = 0; +

SET idle_in_transaction_session_timeout = 0; +

SET client_encoding = 'UTF8'; +

SET standard_conforming_strings = on; +

SET check_function_bodies = false; +

SET client_min_messages = warning; +

SET row_security = off; +

SET search_path = public, pg_catalog; +

SET default_tablespace = ''; +

SET default_with_oids = false; +

CREATE TABLE orasup_test1 ( +

a integer, +

b character varying(200) +

); +

ALTER TABLE orasup_test1 OWNER TO djidba_rw; +

CREATE INDEX idx_b ON orasup_test1 USING btree (b);+

(1 row)

postgres=#

Study note

pg常用大小查询

2020-09-24 没有评论

查出所有数据库大小：

select pg_database.datname, pg_database_size(pg_database.datname) AS size from pg_database order by size desc;

1	select pg_database.datname, pg_database_size(pg_database.datname) AS size from pg_database order by size desc;

查出所有表按大小排序并分离data与index：

SELECT
    table_name,
    pg_size_pretty(table_size) AS table_size,
    pg_size_pretty(indexes_size) AS indexes_size,
    pg_size_pretty(total_size) AS total_size
FROM (
    SELECT
        table_name,
        pg_table_size(table_name) AS table_size,
        pg_indexes_size(table_name) AS indexes_size,
        pg_total_relation_size(table_name) AS total_size
    FROM (
        SELECT ('"' || table_schema || '"."' || table_name || '"') AS table_name
        FROM information_schema.tables
    ) AS all_tables
    ORDER BY total_size DESC
) AS pretty_sizes;

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

SELECT

table_name,

pg_size_pretty(table_size) AS table_size,

pg_size_pretty(indexes_size) AS indexes_size,

pg_size_pretty(total_size) AS total_size

FROM (

SELECT

table_name,

pg_table_size(table_name) AS table_size,

pg_indexes_size(table_name) AS indexes_size,

pg_total_relation_size(table_name) AS total_size

FROM (

SELECT ('"' || table_schema || '"."' || table_name || '"') AS table_name

FROM information_schema.tables

) AS all_tables

ORDER BY total_size DESC

) AS pretty_sizes;

查看索引大小：

select relname as table_name,pg_size_pretty(pg_relation_size(relid)) as table_size, 
indexrelname as index_name,
pg_size_pretty(pg_relation_size(indexrelid)) as index_size from pg_stat_user_indexes 
order by pg_relation_size(indexrelid) desc;

1

2

3

4

select relname as table_name,pg_size_pretty(pg_relation_size(relid)) as table_size,

indexrelname as index_name,

pg_size_pretty(pg_relation_size(indexrelid)) as index_size from pg_stat_user_indexes

order by pg_relation_size(indexrelid) desc;

Study note

小记MySQL的锁和事务

2019-11-20 没有评论

（一）先说明一下定义： 1. 读现象（Read phenomena）： SQL 92标准规定了3种不同的读现象。脏读、不可重复读和幻读。分别解释一下。 1.1 脏读： A dirty read (aka uncommitted dependency) occurs when a transactio […]

Study note

MySQL waiting for metadata lock的分析

2019-09-03 没有评论

处理waiting for metadata lock，需要: 1. 平时打开performance_schema(以下简称PS)的instruments。 2. 查询PS.metadata_locks ，找到状态为PENDING的thread。 3. 查询PS.threads,关联PS.metad […]

Study note

小记scn head room

2019-08-03 没有评论

小记一下前段时间看的scn head room的问题。 1. scn的最大值。scn的表示是SCN_WRAP.SCN_BASE，最大值是 ffff.ffffffff，即65535.4294967295，也就是每当scn_base到ffffffff（或者说4294967295）的时候，scn wrap […]