Microcode SW error with an Intel 8265 / 8275 (rev 78) wireless card

The issue:

Multiple times an hour my wireless connection would drop out for a couple of seconds, then come back up. If this happened while streaming media, most often it would not be noticable because the media buffer would handle the hiccup, but during regular surfing this would lead to disconnects or unreachable pages.

Looking through the dmesg output I found plenty of tracebacks similar to this (MAC address of my router removed):

[   22.603623] iwlwifi 0000:04:00.0: Microcode SW error detected.  Restarting 0x2000000.
[   22.605114] iwlwifi 0000:04:00.0: Start IWL Error Log Dump:
[   22.606304] iwlwifi 0000:04:00.0: Status: 0x00000040, count: 6
[   22.607576] iwlwifi 0000:04:00.0: Loaded firmware version: 36.77d01142.0
[   22.608704] iwlwifi 0000:04:00.0: 0x000014FD | ADVANCED_SYSASSERT          
[   22.609791] iwlwifi 0000:04:00.0: 0x000002F0 | trm_hw_status0
[   22.610872] iwlwifi 0000:04:00.0: 0x00000000 | trm_hw_status1
[   22.611968] iwlwifi 0000:04:00.0: 0x000248EC | branchlink2
[   22.613058] iwlwifi 0000:04:00.0: 0x0003A862 | interruptlink1
[   22.614170] iwlwifi 0000:04:00.0: 0x00000000 | interruptlink2
[   22.615272] iwlwifi 0000:04:00.0: 0x00000082 | data1
[   22.616343] iwlwifi 0000:04:00.0: 0xDEADBEEF | data2
[   22.617448] iwlwifi 0000:04:00.0: 0xDEADBEEF | data3
[   22.618475] iwlwifi 0000:04:00.0: 0x0009505C | beacon time
[   22.619232] iwlwifi 0000:04:00.0: 0x00378063 | tsf low
[   22.620219] iwlwifi 0000:04:00.0: 0x00000000 | tsf hi
[   22.621186] iwlwifi 0000:04:00.0: 0x00000000 | time gp1
[   22.622329] iwlwifi 0000:04:00.0: 0x00378064 | time gp2
[   22.623512] iwlwifi 0000:04:00.0: 0x00000001 | uCode revision type
[   22.624557] iwlwifi 0000:04:00.0: 0x00000024 | uCode version major
[   22.625464] iwlwifi 0000:04:00.0: 0x77D01142 | uCode version minor
[   22.626252] iwlwifi 0000:04:00.0: 0x00000230 | hw version
[   22.626792] iwlwifi 0000:04:00.0: 0x00C89000 | board version
[   22.627331] iwlwifi 0000:04:00.0: 0x003D0108 | hcmd
[   22.627883] iwlwifi 0000:04:00.0: 0x00022080 | isr0
[   22.628434] iwlwifi 0000:04:00.0: 0x00000000 | isr1
[   22.628989] iwlwifi 0000:04:00.0: 0x08201802 | isr2
[   22.629538] iwlwifi 0000:04:00.0: 0x00400080 | isr3
[   22.630588] iwlwifi 0000:04:00.0: 0x00000000 | isr4
[   22.631341] iwlwifi 0000:04:00.0: 0x003C019F | last cmd Id
[   22.631846] iwlwifi 0000:04:00.0: 0x00000000 | wait_event
[   22.632347] iwlwifi 0000:04:00.0: 0x0000D5E7 | l2p_control
[   22.632844] iwlwifi 0000:04:00.0: 0x00000000 | l2p_duration
[   22.633334] iwlwifi 0000:04:00.0: 0x00000000 | l2p_mhvalid
[   22.633835] iwlwifi 0000:04:00.0: 0x00000000 | l2p_addr_match
[   22.634745] iwlwifi 0000:04:00.0: 0x0000008F | lmpm_pmg_sel
[   22.635664] iwlwifi 0000:04:00.0: 0x08081421 | timestamp
[   22.636676] iwlwifi 0000:04:00.0: 0x00004050 | flow_handler
[   22.637518] iwlwifi 0000:04:00.0: Start IWL Error Log Dump:
[   22.638372] iwlwifi 0000:04:00.0: Status: 0x00000040, count: 7
[   22.638861] iwlwifi 0000:04:00.0: 0x00000070 | NMI_INTERRUPT_LMAC_FATAL
[   22.639341] iwlwifi 0000:04:00.0: 0x00000000 | umac branchlink1
[   22.639813] iwlwifi 0000:04:00.0: 0xC0086948 | umac branchlink2
[   22.640278] iwlwifi 0000:04:00.0: 0xC0083B0C | umac interruptlink1
[   22.640745] iwlwifi 0000:04:00.0: 0xC0083B0C | umac interruptlink2
[   22.641211] iwlwifi 0000:04:00.0: 0x00000800 | umac data1
[   22.641693] iwlwifi 0000:04:00.0: 0xC0083B0C | umac data2
[   22.642655] iwlwifi 0000:04:00.0: 0xDEADBEEF | umac data3
[   22.643476] iwlwifi 0000:04:00.0: 0x00000024 | umac major
[   22.644376] iwlwifi 0000:04:00.0: 0x77D01142 | umac minor
[   22.645316] iwlwifi 0000:04:00.0: 0xC088628C | frame pointer
[   22.646226] iwlwifi 0000:04:00.0: 0xC088628C | stack pointer
[   22.647072] iwlwifi 0000:04:00.0: 0x003D0108 | last host cmd
[   22.647786] iwlwifi 0000:04:00.0: 0x00000000 | isr status reg
[   22.648611] iwlwifi 0000:04:00.0: Fseq Registers:
[   22.649236] iwlwifi 0000:04:00.0: 0xE5A33CD7 | FSEQ_ERROR_CODE
[   22.649769] iwlwifi 0000:04:00.0: 0x356D75C8 | FSEQ_TOP_INIT_VERSION
[   22.650294] iwlwifi 0000:04:00.0: 0x2408A24F | FSEQ_CNVIO_INIT_VERSION
[   22.650818] iwlwifi 0000:04:00.0: 0x0000A10B | FSEQ_OTP_VERSION
[   22.651341] iwlwifi 0000:04:00.0: 0x7D82B21C | FSEQ_TOP_CONTENT_VERSION
[   22.651867] iwlwifi 0000:04:00.0: 0xBADED74E | FSEQ_ALIVE_TOKEN
[   22.652391] iwlwifi 0000:04:00.0: 0x21AF7C59 | FSEQ_CNVI_ID
[   22.652914] iwlwifi 0000:04:00.0: 0x32D8E512 | FSEQ_CNVR_ID
[   22.653434] iwlwifi 0000:04:00.0: 0x00000010 | CNVI_AUX_MISC_CHIP
[   22.653957] iwlwifi 0000:04:00.0: 0x0BADCAFE | CNVR_AUX_MISC_CHIP
[   22.654477] iwlwifi 0000:04:00.0: 0x0BADCAFE | CNVR_SCU_SD_REGS_SD_REG_DIG_DCDC_VTRIM
[   22.654998] iwlwifi 0000:04:00.0: 0x0BADCAFE | CNVR_SCU_SD_REGS_SD_REG_ACTIVE_VDIG_MIRROR
[   22.655522] iwlwifi 0000:04:00.0: Collecting data: trigger 2 fired.
[   22.655526] ieee80211 phy0: Hardware restart was requested
[   22.655589] iwlwifi 0000:04:00.0: iwlwifi transaction failed, dumping registers
[   22.656168] iwlwifi 0000:04:00.0: iwlwifi device config registers:
[   22.657016] iwlwifi 0000:04:00.0: 00000000: 24fd8086 00100406 02800078 00000000 dc000004 00000000 00000000 00000000
[   22.657511] iwlwifi 0000:04:00.0: 00000020: 00000000 00000000 00000000 10108086 00000000 000000c8 00000000 000001ff
[   22.658012] iwlwifi 0000:04:00.0: 00000040: 00020010 10008ec0 00190c10 0045e811 10110142 00000000 00000000 00000000
[   22.658509] iwlwifi 0000:04:00.0: 00000060: 00000000 00080812 00000405 00000000 00010001 00000000 00000000 00000000
[   22.659021] iwlwifi 0000:04:00.0: 00000080: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[   22.659511] iwlwifi 0000:04:00.0: 000000a0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[   22.659999] iwlwifi 0000:04:00.0: 000000c0: 00000000 00000000 c823d001 0d000000 00814005 fee00318 00000000 00000000
[   22.660490] iwlwifi 0000:04:00.0: 000000e0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[   22.660984] iwlwifi 0000:04:00.0: 00000100: 14010001 00000000 00000000 00462031 00000000 00002000 00000000 00000000
[   22.661487] iwlwifi 0000:04:00.0: 00000120: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[   22.661988] iwlwifi 0000:04:00.0: 00000140: 14c10003 ff77ed1d 00e18cff 15410018 10031003 0001001e 00481e1f 40a0000f
[   22.662500] iwlwifi 0000:04:00.0: iwlwifi device memory mapped registers:
[   22.663171] iwlwifi 0000:04:00.0: 00000000: 00c89000 00000040 00000000 ba00008b 00000000 00000000 00000010 00000000
[   22.663712] iwlwifi 0000:04:00.0: 00000020: 00000000 08040005 00000230 d55555d5 d55555d5 d55555d5 80008040 041f0042
[   22.664380] iwlwifi 0000:04:00.0: iwlwifi device AER capability structure:
[   22.665072] iwlwifi 0000:04:00.0: 00000000: 14010001 00000000 00000000 00462031 00000000 00002000 00000000 00000000
[   22.665660] iwlwifi 0000:04:00.0: 00000020: 00000000 00000000 00000000
[   22.666231] iwlwifi 0000:04:00.0: iwlwifi parent port (0000:00:1c.6) config registers:
[   22.666902] iwlwifi 0000:00:1c.6: 00000000: 9d168086 00100407 060400f1 00810000 00000000 00000000 00040400 200000f0
///
[   22.677074] iwlwifi 0000:00:1c.6: 00000200: 2201001e 00b0281f 40a0280f
[   22.677731] iwlwifi 0000:04:00.0: iwlwifi root port (0000:00:1c.6) AER cap structure:
[   22.678391] iwlwifi 0000:00:1c.6: 00000000: 14010001 00000000 00010000 00060011 00000000 00002000 00000000 00000000
[   22.679058] iwlwifi 0000:00:1c.6: 00000020: 00000000 00000000 00000000 00000000 00000000 00000000
[   22.679724] iwlwifi 0000:04:00.0: FW error in SYNC CMD PHY_CONTEXT_CMD
[   22.680398] CPU: 2 PID: 1328 Comm: wpa_supplicant Tainted: G           OE     5.4.0-89-generic #100-Ubuntu
[   22.680399] Hardware name: LENOVO 20H9CT01WW/20H9CT01WW, BIOS N1VET56W (1.46 ) 05/17/2021
[   22.680400] Call Trace:
[   22.680405]  dump_stack+0x6d/0x8b
[   22.680415]  iwl_pcie_send_hcmd_sync+0x382/0x390 [iwlwifi]
[   22.680418]  ? wait_woken+0x80/0x80
[   22.680424]  iwl_trans_pcie_send_hcmd+0x34/0xf0 [iwlwifi]
[   22.680430]  iwl_trans_send_cmd+0x55/0xc0 [iwlwifi]
[   22.680438]  iwl_mvm_send_cmd+0x1f/0x40 [iwlmvm]
[   22.680443]  iwl_mvm_send_cmd_pdu+0x51/0x70 [iwlmvm]
[   22.680448]  iwl_mvm_phy_ctxt_apply.constprop.0+0x173/0x240 [iwlmvm]
[   22.680453]  iwl_mvm_phy_ctxt_changed+0x58/0xb0 [iwlmvm]
[   22.680458]  iwl_mvm_change_chanctx+0x10a/0x140 [iwlmvm]
[   22.680477]  ieee80211_recalc_chanctx_min_def+0x1ee/0x340 [mac80211]
[   22.680490]  ieee80211_assign_vif_chanctx+0x21c/0x420 [mac80211]
[   22.680502]  ieee80211_vif_use_channel+0x17b/0x290 [mac80211]
[   22.680514]  ieee80211_prep_channel+0x2be/0x500 [mac80211]
[   22.680524]  ieee80211_prep_connection+0x25f/0x430 [mac80211]
[   22.680536]  ieee80211_mgd_auth+0x19d/0x3d0 [mac80211]
[   22.680548]  ieee80211_auth+0x18/0x20 [mac80211]  Restarting 0x2000000.
[   22.680567]  cfg80211_mlme_auth+0x104/0x210 [cfg80211]
[   22.680579]  nl80211_authenticate+0x284/0x2e0 [cfg80211]
[   22.680582]  genl_family_rcv_msg+0x1b9/0x470
[   22.680584]  ? try_to_wake_up+0x224/0x6a0
[   22.680585]  genl_rcv_msg+0x4c/0xa0
[   22.680587]  ? _cond_resched+0x19/0x30
[   22.680588]  ? genl_family_rcv_msg+0x470/0x470
[   22.680590]  netlink_rcv_skb+0x50/0x120
[   22.680591]  genl_rcv+0x29/0x40
[   22.680592]  netlink_unicast+0x187/0x220
[   22.680593]  netlink_sendmsg+0x222/0x3e0
[   22.680596]  sock_sendmsg+0x65/0x70
[   22.680597]  ____sys_sendmsg+0x212/0x280
[   22.680599]  ___sys_sendmsg+0x88/0xd0
[   22.680600]  ? sock_do_ioctl+0x47/0x140
[   22.680602]  ? __check_object_size+0x4d/0x150
[   22.680605]  ? _copy_to_user+0x2c/0x30
[   22.680606]  ? sock_ioctl+0x24f/0x3c0
[   22.680607]  ? __sys_sendto+0x113/0x190
[   22.680609]  ? __cgroup_bpf_run_filter_setsockopt+0xae/0x2d0
[   22.680611]  ? _cond_resched+0x19/0x30
[   22.680613]  ? aa_sk_perm+0x43/0x170
[   22.680614]  __sys_sendmsg+0x5c/0xa0
[   22.680615]  __x64_sys_sendmsg+0x1f/0x30
[   22.680617]  do_syscall_64+0x57/0x190
[   22.680619]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[   22.680620] RIP: 0033:0x7fe9e8042747
[   22.680622] Code: 64 89 02 48 c7 c0 ff ff ff ff eb bb 0f 1f 80 00 00 00 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 89 54 24 1c 48 89 74 24 10
[   22.680623] RSP: 002b:00007ffd6dbe3fe8 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
[   22.680624] RAX: ffffffffffffffda RBX: 0000558a3dd79700 RCX: 00007fe9e8042747
[   22.680624] RDX: 0000000000000000 RSI: 00007ffd6dbe4020 RDI: 0000000000000006
[   22.680625] RBP: 0000558a3ddcb680 R08: 0000000000000004 R09: 00007fe9e810ab80
[   22.680626] R10: 00007ffd6dbe40f4 R11: 0000000000000246 R12: 0000558a3dd79610
[   22.680626] R13: 00007ffd6dbe4020 R14: 00007ffd6dbe40f4 R15: 0000558a3ddca1b0
[   22.680629] iwlwifi 0000:04:00.0: PHY ctxt cmd error. ret=-5
[   23.196499] iwlwifi 0000:04:00.0: Failed to send MAC context (action:2): -5
[   23.197142] iwlwifi 0000:04:00.0: failed to update MAC 00:e1:8c:77:ed:1d
[   23.198713] iwlwifi 0000:04:00.0: Failed to send MAC context (action:2): -5
[   23.199797] iwlwifi 0000:04:00.0: failed to update MAC 00:e1:8c:77:ed:1d
[   23.200775] wlp4s0: failed to insert STA entry for the AP (error -5)
[   23.200778] iwlwifi 0000:04:00.0: Failed to send MAC context (action:2): -5
[   23.201753] iwlwifi 0000:04:00.0: failed to update MAC 00:e1:8c:77:ed:1d
[   23.203618] iwlwifi 0000:04:00.0: Failed to send quota: -5
[   23.204815] iwlwifi 0000:04:00.0: Failed to send binding (action:3): -5
[   23.206750] iwlwifi 0000:04:00.0: Failed to send MAC context (action:2): -5
[   23.207986] iwlwifi 0000:04:00.0: failed to update MAC 00:e1:8c:77:ed:1d
[   23.209208] iwlwifi 0000:04:00.0: PHY ctxt cmd error. ret=-5
[   23.230555] iwlwifi 0000:04:00.0: Applying debug destination EXTERNAL_DRAM
[   23.362536] iwlwifi 0000:04:00.0: Applying debug destination EXTERNAL_DRAM
[   23.428963] iwlwifi 0000:04:00.0: FW already configured (0) - re-configuring
[   23.559711] wlp4s0: authenticate with [[edited]]
[   23.564136] wlp4s0: send auth to [[edited]] (try 1/3)
[   23.569273] wlp4s0: authenticated
[   23.573418] wlp4s0: associate with [[edited]] (try 1/3)
[   23.575706] wlp4s0: RX AssocResp from [[edited]] (capab=0x431 status=0 aid=1)
[   23.577894] wlp4s0: associated
[   23.651524] IPv6: ADDRCONF(NETDEV_CHANGE): wlp4s0: link becomes ready

Clearly there is an issue with the combination of settings, microcode and hardware that needs to be fixed.

Hardware/firmware

I’m running 5.4.0-89-generic on a Thinkpad T570 with an Intel 8265 / 8275 rev 78 and microcode 36.77d01142.0

lspci -k shows:

04:00.0 Network controller: Intel Corporation Wireless 8265 / 8275 (rev 78)
	Subsystem: Intel Corporation Dual Band Wireless-AC 8265
	Kernel driver in use: iwlwifi
	Kernel modules: iwlwifi

dmesg | grep firmware returns:

[    5.011501] iwlwifi 0000:04:00.0: loaded firmware version 36.77d01142.0 op_mode iwlmvm
[    5.120851] iwlwifi 0000:04:00.0: Allocated 0x00400000 bytes for firmware monitor.

Solution

Reading through forums, one notices this is not uncommon with the Intel iwlwifi modules.

I found a lot of options to handle this, including:

  • Reverting to older microcode by moving the current out of /usr/lib/firmware/ and reverting to an older version. For me this would be going back to v34, then v31 and so on until the issue was gone (not ideal)
  • Going through the upstream and see if newer versions where available upstream (not the case atm)
  • Sending a polite email to linuxwifi@intel.com hoping to get some support (not likely)
  • Get another card (limited to the hw options that Lenovo permits with my default BIOS, and *that* card might also have issues)
  • Changing the module parameters hoping to get improvement.

I opted to go through the last option, and try to limit the amount of changes to what was strictly necessary.

I started by following a recommendation from the Archlinux forum, I created /etc/modprobe.d/iwlwifi.conf (can be any name, as long as it ends in .conf) with the following contents:

options iwlwifi 11n_disable=1 swcrypto=0 bt_coex_active=0 power_save=0
options iwlmvm power_scheme=1 
options iwlwifi d0i3_disable=1 
options iwlwifi uapsd_disable=1 
options iwlwifi lar_disable=1

After a reboot, the issue did not reappear.

However going over dmesg, I found that one of the options (d0i3_disable=1) was not valid, and I set out to reduce the options to the ones (or combination) required to solve the issue.

The current (working) contents of my .conf file:

options iwlwifi 11n_disable=1 swcrypto=0 bt_coex_active=0 power_save=0

Hope this saves someone some time.

pCloud sync issue

I had this crappy issue with with the pCloud Linux (AppImage) client (version linux-x64-prod-v1.9.7).It would get stuck syncing a “Local sync” folder, where Preferences screen as well as the CLI would hang at a certain amount of files and size to be uploaded.


Having no logs to go by (and less than useful technical support on their end), it took me quite a while to find the issue. In the end I found the database where I found some clues as to what was wrong.


The database is called “data.db”, a file in your “~/.pcloud” folder. It is a SQLite database, so install sqlitebrowser to open it, then execute


sqlitebrowser ~/.pcloud/data.db


The “browse data” tab allows you to check the contents of the tables. The “localfolder” table turned out to be what I was looking for. It contains a list of all files that should be synced, with their id, folder, parentfolder, inode …

Contents of the localfolder table.


I noticed that no inode, timestamps or deviceid were given starting at a certain id. That looked interesting. The filename contained an “*” (asterisk) in the filename which doesn’t feel right. I can’t imagine I would have created such a filename intentionally.
So where is this file? The screenshot above shows that it has a folderid of 871638551. Lets search for this id in the most logical table, called “folder”.Using the “Execute SQL” tab we can quickly search for it.

select * from folder where id == 871638551;

and hit CTRL + ENTER.

Result of the SQL query (in the middle)


This returned a row with the name of the file, but a parentfolderid of 288744716. This shows me a row with the folder name that contains the file. You can go up all the way by searching for the parentfolderid of each folder (or write a simple Python script for this), but for me this was enough to show me that the issue was an autogenerated file created by Calibre.


I went online, logged in to my pcloud account, deleted the file, restarted the pcloud client and a couple of minutes later everything was synced. Problem solved.

Finished downloading!

Add warning and tip macros to TiddlyWiki

I am a *very* big fan of TiddlyWiki. I use it to document just about anything I figure out while working on projects, setting up systems, … Anything I would otherwise quickly forget. It makes it easier to add structure to random tidbits of information, and forcing myself to document while I work on something helps to stay focused on a certain goal.

I noticed some of the “official documentation” Tiddlers had options to emphasize tips and (more importantly) warnings, such as these:

While there seems to be an endless list of special tricks one can do with a TiddlyWiki, this was something that seemed really useful so I decided I needed to find out how to recreate them…

Add the macros and stylesheets

In the examples below, replace “Deevee” with any other folder name where you want to gather your own custom macros and stylesheets…

  • Create a new tiddler, name it $:/Deevee/macros
  • Add the tag $:/tags/Macro
  • Past in the following contents:
\define .tip(_) <div class="doc-icon-block"><div class="doc-block-icon">{{$:/core/images/tip}}</div>$_$</div> 
\define .warning(_) <div class="doc-icon-block"><div class="doc-block-icon">{{$:/core/images/warning}}</div>$_$</div>
  • Create another tiddler, name it $:/Deevee/styles
  • add the tag $:/tags/Stylesheet
  • paste in the following contents:
.doc-icon-block {
	border-left: 2px solid <<colour code-border>>;
	margin-left: 3em;
	padding-left: 0.6em;
	position: relative;
}

.doc-block-icon {
	position: absolute;
	left: -3em;
	top: 0.2em;
}

Using the macros

Simply use:

<<.tip “your text”>>

<<.warning “your text”>>

Setting up Visual Studio Code for Python development (on Linux)

The following is a brief guide to reinstalling/setting up Visual Studio Code (VSC) on a new Linux system. The main use is Python development. As such, it is very brief and mostly for my own documentation.

Install VSC

sudo snap install --classic code

Install Python packages

! DO THIS OUTSIDE OF AN ACTIVE VIRTUAL ENVIRONMENT !

  • linting (flake8): python -m pip install pyflakes pycodestyle mccabe flake8
  • Refactoring (rope): python -m pip install rope
  • Formatting (Yapf): python -m pip install yapf
  • Type checking (mypy): python -m pip install -U mypy --user
  • Flake8 Bugbear: python -m pip install flake8-bugbear, then check if installed with flake8 --version (should include flake8-bugbear)
  • Sphinx/RST support: python -m pip install doc8 rstcheck sphinx-autobuild
  • Snooty languageserver: python -m pip install snooty-lextudio

Install and configure plugins

Select interpreter

Add

"python.pythonPath": "/usr/bin/python3",

to $HOME/.config/Code/User/settings.json

Set up linting

List of possible linters and how to configure them

Open user settings and

  • set Python > linting: flake8 path, to C:\Python39\Scripts\flake8.exe
  • Add max-line-length=99 to Python > Linting: Flake8 Args
  • Enable flake8

or add

    "python.linting.flake8Path": "C:\\Python39\\Scripts\\flake8.exe",
    "python.linting.flake8Args": [
        "max-line-length=99"
    ],
    "python.linting.flake8Enabled": true,

to %APPDATA%\Code\User\settings.json (Windows) or $HOME/.config/Code/User/settings.json (Linux)

MyPy:

  • Install package

python -m pip install -U mypy --user

  • Enable
    "python.linting.mypyEnabled": true,
    "python.linting.mypyArgs": [],
    "python.analysis.typeCheckingMode" :"basic",
    "python.linting.mypyPath": "C:\\Users\\dieterv\\AppData\\Roaming\\Python\\Python39\\Scripts\\mypy.exe",

to ~/.config/Code/User/settings.json

more mypy info

Autodocstring

Add the following to your settings.json

"autoDocstring.docstringFormat": "google",
"autoDocstring.guessTypes": true,
"autoDocstring.generateDocstringOnEnter": true,
"autoDocstring.includeExtendedSummary": true,
"autoDocstring.quoteStyle": "\"\"\"",

Home Server – setting up ZFS (revisited)

Finally, after two years of renovations and two children, I am revisiting the idea of a small home server. I don’t expect it to do much:

  • Media server
  • File backup
  • Some docker containers
  • Grafana
  • Possibly pfSense or another firewall

Whenever I’m accessing a remote system (even when using SSH in my internal network) I prefer to start a TMUX session. That way, if I get disconnected (for instance) during the middle of a package upgrade, that process just continues and I can reconnect at a later time without issues (if tmux is not yet installed or you’re not using it yet, I highly recommend reading up on it and taking a bit of time to get used to the basic concepts. I used this quickstart a couple of years ago).

tmux new -s sessionname

Install the required zfs package:

sudo apt update && sudo apt install zfsutils-linux

Then find out which drives you want to use:

sudo fdisk -l

In my case, I wanted to use the two 1.8GB drives (/dev/sdb and /dev/sdc) in a mirror configuration. I don’t have a hundred movies to fill a drive, most space will be taken up by photos and system images.

sudo zpool create mypool mirror /dev/sdb /dev/sdc

Check the pool status to verify that it is as intended:

zpool status

The pool automatically gets mounted on /mypool with root:root as owner. I changed the owner/group to my own user

sudo chown <user>:<user> /mypool/ (replace <user>)

Using MicroPython (PyBoard V1.1) with Visual Studio Code

Warning

This still isn’t working as I think it should. Especially the linting and autocomplete still doesn’t work (leaving only the PyMakr module which works partly, in fact). For example: “import pyb” still does not add autocompletion when typing something like “s = pyb.<CTRL + SPACE>”. Most likely this is because I used a combinations of tools that shouldn’t be used together. Need to spend more time on this…

Intro

I’ve recently started using a PyBoard for a project that I am working on. Writing code in an IDE, saving the files, then manually copying them to the board to test them gets tedious rather quickly. If you’re using Visual Studio Code, there are options to do almost everything from within the IDE itself. Getting things to work as they should took a bit of effort and headscratching. What I needed to do is documented below.
Links to the sources of my information is at the bottom.

Install libraries/addon

  • pip install --upgrade pip --user
  • pip install --upgrade micropy-cli --user
  • pip install micropy-cli[create_stubs]
  • pip install rshell --upgrade
  • To prevent “Source directory /pyboard/stubs does not exist.” after creation open pybwrapper.py, change line 63 from
    pyb_path = f"{self.pyb_root}{_path}"
to:
    pyb_path = f"/flash/{_path}"

and save the file.

  • Inside VSC, install the Pymakr plugin
  • Install node.js (see link below, and reboot afterwards)

Create the stub yourself…

(This did not work properly for me. See below for another option.)

  • Connect the PyBoard over USB
  • Find out the serial port number (represented from here on by comXX)
  • Run
    micropy stubs create comXX

    (replace XX by com port number) -> this takes a couple of minutes

  • the result will be copied to ~/.micropy/stubs (in my case in a sub directory “pyboard-1.13.0”

…Or get them from someone else

I got mine from Josverl on Github. See his read.me for more information as well.

  • Pick a directory that will hold the files. Clone the repo:
    git clone git@github.com:Josverl/micropython-stubs.git
  • create a symlink in your project folder:

    mklink /d all-stubs <directory holding the stubs>
  • create a settings.json file in your project folder containing (change the references to the stubs you need, in the desired order):
{
    "python.linting.enabled": true,
    "python.linting.pylintEnabled": true,
    "python.languageServer": "Microsoft",
    "python.jediEnabled": false,
    "python.autoComplete.extraPaths": [
         "all-stubs/cpython_core",
         "all-stubs/pyboard_1_13_95",
         "all-stubs/mpy_1_13-nightly_frozen/GENERIC",
         "all-stubs/cpython_pyboard"
     ],
     "python.autoComplete.typeshedPaths": [
          "all-stubs/cpython_core",
         "all-stubs/pyboard_1_13_95",
          "all-stubs/mpy_1_13-nightly_frozen/GENERIC",
          "all-stubs/cpython_pyboard"
     ],
     "python.analysis.typeshedPaths": [
          "all-stubs/cpython_core",
          "all-stubs/pyboard_1_13_95",
          "all-stubs/mpy_1_13-nightly_frozen/GENERIC",
         "all-stubs/cpython_pyboard"
    ]
}
  • Copy the .pylintrc file from the sample folder and adapt the init-hook line to reflect the chosen folders (in last step):
[MASTER]
init-hook='import sys;sys.path[1:1] = ["all-stubs/cpython_core", "all-stubs/pyboard_1_13_95", "all-stubs/mpy_1_13-nightly_frozen/GENERIC", "all-stubs/cpython_pyboard",];'
  • Restart VSC:
Developer: Reload Window

Set up your project (in VSC terminal, with project open)

  • To check that your stub (config file for your board) has been installed run
    micropy stubs list 
  • If not, run
    micropy stubs add <path to stub> 

    (where the path is the directory that contains the info.json file.)

  • run
    micropy init <project name>
  • Select what to generate (‘a’ selects all)
  • select which stub to use (the one you just created)
  • A subdirectory will created with <project name> you just specified
  • Open the pymakr.conf file in the subdirectory <project name>. Change
    "address":"192.168.4.1" 

    to your comXX. To do this system wide for Pymakr, open the command palette (CTRL+SHIFT+P) and select/type “Pymakr > Global settings”

  • In the same config file, I had to set “auto_connect”: false to resolve some vague
    "> Failed to connect (Error: Port is not open). Click here to try again." 

    errors when connecting to the board. Setting this to “true” the following behavior: “ignores any ‘address’ setting and automatically connects to the top item in the serial port list”

  • See here for more explanation on the other options

Using the combination

  • To add package dependencies to your project:
    micropy install <PACKAGE_NAMES>
  • Functionality in the Pymakr addon can be used by opening the command palette (CTRL+SHIFT+P) and typing Pymakr. There are options to (dis)connect, run the current file, up/download the whole project, get the FW version,…
  • Note the new terminal “Pymakr Console”:
    Pymakr console
  • And the “Pymakr toolbar”:
    Pymakr toolbar

Shortcuts (from the Pymakr addon info)

ctrl-shift-c : (Re)connect
ctrl-shift-g : Global settings
ctrl-shift-s : Synchronize project
ctrl-shift-r : Run current file
ctrl-shift-enter : Run current Line

References

Installing OpenVPN on Ubuntu 12.04

The company I have been working for for a couple of months has an OpenVPN server running. Recently I wanted to install the client software on my Windows 7 desktop and my Linux laptop.

Our IT guy kindly supplied me with a executable to install and configure the client under Windows and a config file and certificates for Linux.

Windows 7 worked immediately but under Linux I had to do some extra steps to get it working properly. I’ll document them here for future reference and maybe other people running in to the same issues.

I had previously installed OpenVPN under linux and hoped the config file would be all I needed. When running

openvpn –config configfile.ovpn

I got the error message:

Options error: Unrecognized option or missing  parameter(s) in configfile.ovpn:11: verify-x509-name (2.2.1)
Use –help for more information.

Some digging around showed the “verify-x509-name” command was implemented starting at OpenVPN version 2.3. Using

apt-get install openvpn

and the default repositories in Ubuntu the installed version was 2.2.1

To get the latest version, I uninstalled the current installed version using

apt-get purge openvpn

then downloaded and untarred the latest version from the community downloads (at this time 2.3.6)

wget http://swupdate.openvpn.org/community/releases/openvpn-2.3.6.tar.gz

tar -zxvf ./openvpn-2.3.6.tar.gz

Installing should work this way:

cd ./openvpn-2.3.6

./config && sudo make && sudo make install

but I ran into dependency problems. Some error messages came up regarding the LZO package.

configure: error: lzo enabled but missing

I found this page by Droidzone describing the dependencies and some useful information on how to find missing dependencies during future installs. Worth a read.

In short:

sudo apt-get install libssl-dev liblzo2-dev libpam0g-dev

will install the necessary dependencies. After this the installation went fine:

openvpn –version

OpenVPN 2.3.2 x86_64-unknown-linux-gnu [SSL (OpenSSL)] [LZO] [EPOLL] [eurephia] [MH] [IPv6] built on Dec 24 2014

Wake-on-lan (WOL) bash script on router running dd-wrt

After setting up my router to run dd-wrt I’ve used PuTTy to ssh into the router to start my desktop machine to work on it through a RDP session.

After spending months looking up the correct MAC address in my phone and typing the commands by hand I finally took a bit of time to write a small bash script on the router so that I could simply run the script each time I needed it.

On my router (WRT54GL) there’s a bit of space in the /jffs folder that’s usable for storing scripts.

To write a script named “wol” go to the correct directory and type:

vi wol

VI opens in command mode. To start editing press the INSERT key. At the bottom left of the screen a “I” will be displayed. The editor is now in editing mode. I just wrote:

#!/bin/ash

echo “Waking up 19\” system”

/usr/sbin/wol -i 192.168.100.255 -p 9 xx:xx:xx:xx:xx:xx

echo “Done”

Where xx:xx:xx:xx:xx:xx is the MAC address of my network card.

To get back in command mode press ESCAPE.

Then to save the file and exit type:

:wq and ENTER. (the semicolon starts the command, wq stands for write and quit).

This creates the script, the only thing left is to make is executable:

chmod 700 wol

From now on, the script can be started by going to the correct directory and typing:

./wol

The script will be called and run the commands. If there are any error messages they will be displayed.

Ubuntu Server – setting up ZFS (part 1)

For a while I’ve wanted to learn about Linux, the way the OS works, what can be done with it, if it’s reasonable to change to it as my main operating system.

As well as dual booting Windows 7 and Ubuntu 12.04 on my laptop I thought I should try and set up my home server running a Linux distro. This way I’d be forced to learn my way around the command line, setting up a system to run smoothly without the need for much maintenance.

The things I wanted to try to set up were:

  • Some sort of NAS, with the option of mounting it as a samba drive over my internal network and using some sort of FTP that has to be secure when accessing the shares from outside of my home network.
  • I really want to give Owncloud a try. Not just as an alternative to Dropbox, but also as a central calendar and contact database. Over WebDAV that should be possible. I’ll have to get a basic Apache setup running to do this.
  • I’d like to try out the ZFS file system as I have no previous experience with it.
  • To have an openVPN server running at home would be nice when working abroad for longer periods of time (tunneling email, accessing home network, …)
  • Maybe a Socks proxy (?)
  • Keeping it all secure

As I have some experience with Ubuntu desktop I picked Ubuntu Server to start with. As I go along setting up the system I might end up deciding to pick something else and start over, but as I take this as a learning experience that’s fine.

So I started downloading the Ubuntu Server 12.04 AMD64 iso and made a bootable USB stick to install the system.

During the installation I didn’t change much from the standard settings. I partitioned the disk before installing the OS but nothing else. As I knew I was going to need the Apache and MySQL packages for Owncloud I picked the LAMP bundle for installation as well.

After installation one of the first things to do was to get a zfs pool up and running. I had picked two Western Digital RED series 2 TB disks as my data disks as they are made to be running for extended periods of time and one Seagate Spinpoint 320 GB 2.5″ drive for the OS and swap. This should give me a good balance between power and storage. The 2.5″ will be set up to go into idle really quick to save power.

Installing Ubuntu ZFS

To install the ubuntu-zfs package the ppa:zfs-native/stable repository needs tot be added to the system.

Apparently the add-apt-repository command wasn’t installed yet on the 64 bit version on my system. There are two options:

  1. Install the python-software-properties package
  2. manually add the repositories to /etc/apt/sources.list

As I will need to add more ppa’s later on I chose the first option.

A quick sudo apt-get install ubuntu-zfs and some 170 Meg later the package was build and installed.

Creating a ZFS pool

sudo parted followed by print devices displays a list of devices with their storage capacity:

(parted) print devices
/dev/sda (320GB)
/dev/sdb (2000GB)
/dev/sdc (2000GB)

/dev/sdb and /dev/sdc are my two disks that will be used for a mirrored ZFS pool.

sudo zpool create zfs mirror /dev/sdb /dev/sdc creates a mirrored pool named zfs using the two disks in a mirrored configuration. Instead of residing in the /media or /mount directory, it can be found in the root of the file system.

sudo zpool list confirms that the pool exists:

NAME   SIZE  ALLOC   FREE    CAP  DEDUP  HEALTH  ALTROOT
zfs   1.81T   496K  1.81T     0%  1.00x  ONLINE  –

The next step will be to test the pool, writing data to it, unplugging one of the disks, formatting it, adding it to the pool again and check if everything works as fine as it should.

Before doing that I’ll set up a samba share on the zfs directory. This will provide me with a real-world situation where I can access the pool from another computer in my network.

Recent projects

So.. After a diving trip it’s about time to write up on two recent projects:

  1. Home server: As I wanted to get into Linux for a long time and also wanted to set up a home server for VPN/SSH/FTPES/firewall/my own Exchange replacement and sharing of music and movies in my own network (and remote, of course), this seemed like a good time to dive in. Starting with a generic Ubuntu server package without GUI I’ve started setting up a low power system that I can leave on permanently without having to worry about it too much. A good opportunity to read up on current computer hardware as well.
  2. Dive light: a while ago I bought a halogen dive light with the idea of using the housing for a diy LED dive light. Benefits: light output, can choose my own beam characteristics, longer runtime. Drawbacks: colour reproduction. Pretty much still a work in progress, but I’ve done about 20 dives with a system with 2 XM-L’s driven at about 900 mA’s and easily lights up the beachfront after a night dive. Also: first time working with a lathe.

More about these two soon.