分类默认分类下的文章

VSC正则表达式替换

作者: 拾月
时间: 2023-07-19
分类: 默认分类
评论

替换空白行（含有空白字符的行也替换）

#填入查找框
^\s*(?=\r?$)\n

替换空白行（）

#填入查找框，表示查找连续的2 或更多个换行符
\n{2,}
#填入替换框，表示用一个换行符替换连续的多个换行符
\n

不以某字符串str开头

^(?!str)

不以某字符串str结尾

(?<!str)$

其他几个 ? 元字符用法

(?:str)   非捕获组

(?=str) 肯定式向前查找

(?!str) 否定式向前查找

(?<=str) 肯定式向后查找

(?<!str) 否定式向后查找

Visual Studio Code（VSC）编辑器的使用分组的正则表达式替换

作者: 拾月
时间: 2022-05-30
分类: 默认分类
评论

For beginners, the accepted answer is correct, but a little terse if you're not that familiar with either VSC or Regex.

So, in case this is your first contact with either:

To find and modify text,

In the "Find" step, you can use regex with "capturing groups," e.g. I want to find (group1) and (group2), using parentheses. This would find the same text as I want to find group1 and group2, but with the difference that you can then reference group1 and group2 in the next step:

In the "Replace" step, you can refer to the capturing groups via $1, $2 etc, so you could change the sentence to I found $1 and $2 having a picnic, which would output I found group1 and group2 having a picnic.

Notes:

Instead of just a string, anything inside or outside the () can be a regular expression.
$0 refers to the whole match

来源地址

使用SDK和imagebuilder编译自己使用的openwrt固件

作者: 拾月
时间: 2019-11-25
分类: 默认分类,Linux Tips
评论

系统

本次使用的是ubuntu18.04LTS mini,其他版本可能编译出来的固件会不停的重启

依赖安装

sudo apt-get install subversion build-essential libncurses5-dev zlib1g-dev gawk git ccache gettext libssl-dev xsltproc zip python

下载对应的sdk和imagebuilder

下载链接见官网

编译自己需要的包

adbyby
luci-app-adbyby-plus
frpc
luci-app-frpc
luci-app-ttyd
vlmcsd
luci-app-vlmcsd
luci-app-vsftpd
vsftpd-alt

编译步骤
解压下载的SDK，并切换到sdk的package目录，然后下载源码

svn co https://github.com/coolsnowwolf/lede/trunk/package/lean/luci-app-adbyby-plus
svn co https://github.com/coolsnowwolf/lede/trunk/package/lean/adbyby
svn co https://github.com/coolsnowwolf/lede/trunk/package/lean/luci-app-frpc
svn co https://github.com/coolsnowwolf/lede/trunk/package/lean/frpc
svn co https://github.com/coolsnowwolf/lede/trunk/package/lean/luci-app-ttyd
svn co https://github.com/coolsnowwolf/lede/trunk/package/lean/vsftpd-alt
svn co https://github.com/coolsnowwolf/lede/trunk/package/lean/luci-app-vsftpd
svn co https://github.com/coolsnowwolf/lede/trunk/package/lean/autosamba
svn co https://github.com/coolsnowwolf/lede/trunk/package/lean/luci-app-vlmcsd
svn co https://github.com/coolsnowwolf/lede/trunk/package/lean/vlmcsd

更新上面package里需要的依赖feeds

./scripts/feeds update -a
./scripts/feeds install luci uci libubox lua libev

生成需要编译包的shell script，并修改删除不要的一些行，然后运行这个脚本

ll package/ |awk '{print "make package/"$9"compile V=99"}' >willCompile.sh
chmod +x a.sh
bash ./willCompile.sh

将编译好的ipk，复制到imagebuilder解压后生成的目录下的packages目录里

find . -name *.ipk -exec mv {} imagebuilder/packages/ \;

切换到imagebuilder目录，修改repositories.conf 文件，添加翻墙需要的包的仓库

src/gz openwrt_dist http://openwrt-dist.sourceforge.net/packages/base/mips_24kc
src/gz openwrt_dist_luci http://openwrt-dist.sourceforge.net/packages/luci

注意路由器的架构，修改对应的mips_24kc

[root@NETGEAR:/root]# opkg print-architecture | awk '{print $2}'  # 查看路由器架构
all
noarch
mips_24kc

http://www.zhangjun.sh.cn/index.php/other/openwrt-ss-chinadns.html

NETGEAR wndr4300v1 还要修改 target/linux/ar71xx/image/legacy.mk 文件，使用完整的128MB空间（已测试于OpenWrt 18.06.5）

sed -i s/'23552k(ubi),25600k@0x6c0000(firmware)'/'120832k(ubi),122880k@0x6c0000(firmware)'/ target/linux/ar71xx/image/legacy.mk

https://www.right.com.cn/forum/thread-521490-1-1.html
https://forum.openwrt.org/t/netgear-wndr4300-extra-flash-space/38183

添加必备的一些设置，比如自动挂载U盘脚本，设置lan口ip、科学上网设置等，这些可以在imagebuilder目录下新建 files/etc/uci-defaults 目录，然后在其中添加对应功能的uci命令的shell脚本，比如：

#!/bin/sh

uci add_list dhcp.@dnsmasq[0].confdir=/etc/dnsmasq.d
uci add_list dhcp.@dnsmasq[0].cachesize=10000
uci commit dhcp

最后，开始编译固件
NETGEAR wndr4300v1

make image PROFILE=WNDR4300V1 PACKAGES="-dnsmasq dnsmasq-full -wpad-mini wpad curl coreutils-base64 \
wget ca-certificates ca-bundle iptables-mod-tproxy ChinaDNS luci-app-chinadns shadowsocks-libev luci-app-shadowsocks simple-obfs \
luci-app-adbyby-plus luci-app-frpc luci-app-ttyd luci-app-vsftpd luci-i18n-adbyby-plus-zh-cn luci-i18n-ttyd-zh-cn luci-i18n-vsftpd-zh-cn \
luci luci-base luci-i18n-base-zh-cn \
luci-i18n-firewall-zh-cn block-mount \
kmod-fs-cifs kmod-fs-ext4 kmod-fs-msdos kmod-fs-vfat ntfs-3g \
kmod-nls-base kmod-nls-cp437 kmod-nls-cp936 kmod-nls-iso8859-1 kmod-nls-utf8 \
samba36-hotplug luci-app-samba luci-i18n-samba-zh-cn samba36-server \
kmod-usb-core kmod-usb-ohci kmod-usb-storage kmod-usb-storage-extras kmod-usb-uhci kmod-usb2 kmod-usbmon usbutils \
hd-idle luci-app-hd-idle luci-i18n-hd-idle-zh-cn \
luci-app-wol luci-i18n-wol-zh-cn \
minidlna luci-app-minidlna luci-i18n-minidlna-zh-cn \
wifischedule luci-app-wifischedule luci-i18n-wifischedule-zh-cn \
fdisk lrzsz openssh-sftp-server \
vlmcsd luci-app-vlmcsd luci-i18n-vlmcsd-zh-cn" FILES=files/

TP-LINK WR720N

make image PROFILE=tl-wr720n-v3 PACKAGES="-dnsmasq dnsmasq-full -wpad-mini wpad curl coreutils-base64 \
wget ca-certificates ca-bundle iptables-mod-tproxy ChinaDNS luci-app-chinadns shadowsocks-libev luci-app-shadowsocks simple-obfs \
luci-app-adbyby-plus luci-app-frpc luci-app-ttyd luci-app-vsftpd luci-i18n-adbyby-plus-zh-cn luci-i18n-ttyd-zh-cn luci-i18n-vsftpd-zh-cn \
luci luci-base luci-i18n-base-zh-cn \
luci-i18n-firewall-zh-cn block-mount \
kmod-fs-cifs kmod-fs-ext4 kmod-fs-msdos kmod-fs-vfat ntfs-3g \
kmod-nls-base kmod-nls-cp437 kmod-nls-cp936 kmod-nls-iso8859-1 kmod-nls-utf8 \
autosamba luci-app-samba luci-i18n-samba-zh-cn samba36-server \
kmod-usb-core kmod-usb-ohci kmod-usb-storage kmod-usb-storage-extras kmod-usb-uhci kmod-usb2 kmod-usbmon usbutils \
hd-idle luci-app-hd-idle luci-i18n-hd-idle-zh-cn \
luci-app-wol luci-i18n-wol-zh-cn \
minidlna luci-app-minidlna luci-i18n-minidlna-zh-cn \
wifischedule luci-app-wifischedule luci-i18n-wifischedule-zh-cn \
fdisk lrzsz openssh-sftp-server \
vlmcsd luci-app-vlmcsd luci-i18n-vlmcsd-zh-cn" FILES=files/

最后，4300刷机步骤

文件名：openwrt-15.05-ar71xx-nand-wndr4300-ubi-factory.img

推荐直接 tftp 刷机：
1.电脑设置静态 IP：
IP：192.168.1.2
子网掩码：255.255.255.0
默认网关：192.168.1.1

2.将电脑连接至路由器任一 LAN 口

3.在 Windows 控制面板 -> 程序 -> 启用或关闭Windows功能中，开启 TFTP 客户端

4.关闭路由器电源，使用牙签(或其他类似物)捅住路由器背面的 reset 按钮不放，打开路由器电源，待电源指示灯呈绿色并有规律闪动即可松开 reset 按钮

5.传送固件，打开命令行窗口，使用
tftp -i 192.168.1.1 put <路径>/openwrt-15.05-ar71xx-nand-wndr4300-ubi-factory.img
传送固件至路由器，出现如下提示则固件传送成功：
传输成功: 3 秒 12451969 字节，4150656 字节/秒

6.静待路由器重启并完成固件刷新，切不可着急强制切断电源

7.待确认路由器重启成功后(比如可以登录http://192.168.1.1)，直接关闭路由器电源30秒以上，然后再打开电源，否则可能出现 5G WiFi 不可用的问题。

科学上网设置

http://www.zhangjun.sh.cn/index.php/other/openwrt-ss-chinadns.html
所有设置好，并测试完毕，可以备份设置，再次编译并刷机，这样，恢复出厂设置也不要设置很多地方。

其他事项

查看已安装的包

echo $(opkg list-installed | sed -e "s/\s.*$//")

常用设置

uci del dhcp.lan.dhcp_option
uci add_list dhcp.lan.dhcp_option='6,223.5.5.5,223.6.6.6'

# /etc/config/network

uci del network.lan.dns
uci add_list network.lan.dns='223.5.5.5'
uci add_list network.lan.dns='223.6.6.6'

编译的默认设置

xxx@debian:~/openwrt/openwrt-imagebuilder-22.03.5-ramips-mt7620.Linux-x86_64$ cat files/etc/uci-defaults/abc.sh 
#!/bin/sh


uci add_list dhcp.@dnsmasq[0].confdir=/etc/dnsmasq.d
uci add_list dhcp.@dnsmasq[0].cachesize=10000
uci commit dhcp

uci set luci.main.lang=zh_cn
uci commit luci

uci set system.@system[0].hostname='Lenovo'
uci set system.@system[0].timezone=CST-8
uci set system.@system[0].zonename=Asia/Shanghai
uci commit system

uci del dhcp.lan.dhcp_option
uci add_list dhcp.lan.dhcp_option='6,223.5.5.5,223.6.6.6'

# /etc/config/network

uci del network.lan.dns
uci add_list network.lan.dns='223.5.5.5'
uci add_list network.lan.dns='223.6.6.6'
uci set network.lan.ipaddr='192.168.99.1'
uci commit

sed -i '/option disabled/d' /etc/config/wireless
sed -i '/set wireless.radio${devidx}.disabled/d' /lib/wifi/mac80211.sh
sed -i 's/OpenWrt/Lenovo/' /lib/wifi/mac80211.sh

cat >> /etc/profile <<EOF

export PS1='[\[\033[35;1m\]\u\[\033[0m\]@\[\033[31;1m\]\h\[\033\[0m\]:\[\033[32;1m\]$PWD\[\033[0m\]]\$ '

alias df='df -Th'
alias free='free -m'
alias la='ll -A'
alias ll='ls -alh --color=auto'
alias ls='ls --color=auto'

[ -x /bin/more ] || alias more=less
[ -x /usr/bin/vim ] && alias vi=vim || alias vim=vi
alias ll='ls -alF --color=auto'
EOF

exit 0

无线密码

# /etc/config/wireless
uci set wireless.radio0.cell_density='0'
uci set wireless.default_radio0.ssid='Lenovo-5G'
uci set wireless.default_radio0.encryption='psk2'
uci set wireless.default_radio0.key='---------'
uci set wireless.radio1.cell_density='0'
uci set wireless.default_radio1.ssid='Lenovo'
uci set wireless.default_radio1.encryption='psk2'
uci set wireless.default_radio1.key='----------'

pipenv使用说明

作者: 拾月
时间: 2019-11-07
分类: 默认分类
评论

安装
pip3 install pipenv

新建项目虚拟环境的步骤
1.创建项目目录，并进入（cmd）
2.项目初始化

pipenv --python 3

3.安装项目所需的包，会生成两个文件pipfile、pipfile.lock

pipenv install flask flask_mysqldb
pipenv install requests==2.13.0 # 指定包的版本

4.进入虚拟环境的方式

pipenv shell

5.在虚拟环境运行命令

pipenv run xxx
eg：pipenv run pip list

6.查看虚拟环境相关信息，项目所绑定的虚拟目录

pipenv --venv

7.查看项目各种包的依赖关系

pipenv graph

8.安装开发环境测试包，（不部署到生产环境），注意，在Pipfilede [dev-packages]

pipenv install --dev requests

9.团队项目环境共享方法，直接分享pipfile这个文件即可。然后：

pipenv install       #安装初始化项目依赖包，不包括开发环境的依赖包
pipenv install --dev #包括开发环境依赖包

10.删除虚拟环境

pipenv --rm

11.代码运行

pipenv run python main.py

12.运行脚本，脚本定义在pipfile中的[scripts]段，比如

[scripts]
start = "python main.py"
test = "pytest"
list = "pip list"

运行：pipenv run start  #这里的start，就是上面定义的。

13.从 requirements.txt 导入环境

pipenv install -r path/to/requirements.txt

14.生成 requirements.txt

pipenv lock -r     # 生成requirements.txt文件
pipenv lock -r -d  # 生成dev-packages的requirements.txt文件

15.常见命令介绍

pipenv install                  # 安装包
pipenv shell                  # 激活当前项目的虚拟环境
pipenv install pytest --dev   # 安装开发依赖包
pipenv graph                  # 图形显示包依赖关系
pipenv lock                      # 生成lockfile
pipenv uninstall --all          # 删除所有的安装包

16.bash的shell补全，添加下面语句到.bashrc或.bash_profile

eval "$(pipenv --completion)"

https://crazygit.wiseturtles.com/2018/01/08/pipenv-tour/

Python多线程抓取小说并保存成文件

作者: 拾月
时间: 2019-09-10
分类: 默认分类
评论

#!/usr/bin/python
# -*- coding: utf-8 -*-

import requests,time,re
from lxml import etree
from decimal import Decimal
from unicodedata import normalize
from multiprocessing import Pool,cpu_count

headers = {
        'User-Agent':'Mozilla/5.0 (Windows NT 5.1; rv:52.0) Gecko/20100101 Firefox/52.0',
        'Referer':'https://www.vodtw.com/',
    }

def getHtml(url,headers=headers):
    r = requests.get(url,headers=headers,proxies=None)
    return r.content

def useXpath(html):    
    html = etree.HTML(html)
    urls = html.xpath("//dl/dd/a/@href")
    return urls

def getContent(url):
    url = 'https://www.vodtw.com/book/516/' + url
    html = getHtml(url)

    # 每个章节只显示一部分，另外一部分通过xhr异步请求。
    reg_p_bufUrl = "bufurl='(.*?)'"
    matches = re.findall(reg_p_bufUrl, html.decode('utf-8'), re.MULTILINE)
    nextUrl = 'https://www.vodtw.com' + matches[0]

    html = etree.HTML(html)
    p = html.xpath("//*[@id='content']/text()")
    title = html.xpath("//h1/text()")
    print(f"正在抓取 {url} {title[1]}")
    content = ''
    for item in p:
        content += normalize("NFKD",item) + '\n'

    temp = getHtml(nextUrl).decode('utf-8').replace('<br><br>','\n')
    content = content[:-1] + temp
    # print(content)
    return {"title":title[1],"content":content,"url":url}

if __name__ == "__main__":
    start = time.time()
    url = 'https://www.vodtw.com/book/516/'
    html = getHtml(url)
    mulu = useXpath(html)
    filename = '寒门枭士.txt'
    cpus = cpu_count()
    pool = Pool(cpus)
    res_list = []
    i = 0
    allcontents = []
    for url in mulu[18:]:
        i =i+1
        res = pool.apply_async(func=getContent, args=(url,))
        res_list.append(res)

        if i %10 == 0:            
            # 每10章写入文件一次。
            for res in res_list:
                content = res.get()
                if content:
                    allcontents.append(content)
            
            for x in allcontents:
                with open(filename,'a',encoding='utf-8') as f:
                    f.write(x["title"] + '\n')
                    f.write(x["content"] + '\n')
            allcontents.clear()
            res_list.clear()
            time.sleep(3)
    # 进程池关闭
    pool.close()

    # 等待所有进程结束
    pool.join()

    for x in allcontents:
        with open(filename,'a',encoding='utf-8') as f:
            f.write(x["title"] + '\n')
            f.write(x["content"] + '\n')  

    print(f"抓取完毕，总共耗时：{Decimal((time.time()-start)//60).quantize(Decimal('0'))}分{Decimal((time.time()-start)%60).quantize(Decimal('0.00'))}秒")