Replication

On Primary:
– Lock tables:
FLUSH TABLES WITH READ LOCK;
keep this window open

– on another screen:
SHOW MASTER STATUS;
mysql-bin.000088 | 338004

On Secondary:
– stop mysql

On primary:
rsync -Sa –progress –exclude=mastername* –exclude=master.info –exclude=relay-log.info /var/lib/mysql/* <replicant>:/var/lib/mysql

On Secondary:
– start mysql
– start replication:
CHANGE MASTER TO
MASTER_HOST=’ip’,
MASTER_USER=’mysql_backup’,
MASTER_PASSWORD='<pass>’, MASTER_LOG_FILE=’mysql-bin.000088′, MASTER_LOG_POS=338004

On primary:
– close original window

Varnish A/B Testing

Cloudflare doesn’t cache any of our PHP files so don’t need to worry about that. Cloudflare connects to our nginx SSL reverse proxy.

nginx has a split_clients directive, so could potentially add header there:

Performing A/B Testing with NGINX and NGINX Plus

The SSL reverse proxy connects to a varnish server on the same system.

Varnish can also set the a/b testing header:
https://info.varnish-software.com/blog/live-ab-testing-varnish-and-vcs

Where ever the header is set, Varnish will need to Vary based on that header.

Varnish then connects to nginx, which then connects to the php-fpm server(s). At this point, there needs to be a different wordpress instance loaded, and there are a couple different ways.

The main idea I have now is to bind mount different plugin and theme directories on top of a base directory. The base directory contains the basic wordpress install, and the images, etc (basically everything not themes/plugins). One easy way to do this is with containers, but you should be able to do it outside containers as well.

No Containers Method:
– bind mount base directory to branch directory (ie /vhosts/base -> /vhosts/test-a)
– bind mount plugins and theme directories on top (ie /usr/src/test-a/plugins -> /vhosts/test-a/wordpress/wp-content/plugins)
– map header to root directories in nginx: (needs to be in http block)
https://serversforhackers.com/c/nginx-mapping-headers
#should use mapping so that there is always a default (don’t trust header)
map $http_ab_group $ati_ab_dir {
default “all-that-is-interesting”;
a “test-a”;
}
server {
server_name allthatsinteresting.com;
server_name www.allthatsinteresting.com;
root /vhosts/$ati_ab_dir;
}

Containers Method:
– switch nginx config to use containers (basically need to use a reverse proxy instead of fastcgi)
– which reverse proxy to use would be mapped in from the header
– containers would have the base dir, plugins dir, and themes dir mounted as volumes

Containers method would require more work, and the worker pool will be segmented among the php-fpm backends, so it’s more difficult to size the max workers per container. Like say the server has memory to handle 128 workers, if you have 3 separate worker pools, do you assume that all 3 pools will be maxed, or do you oversubscribe since chances are the load will be uneven? It’s simpler and safer to just have a single PHP backend. (not to mention the added complexity of Docker)

64-bit relocatable addresses

I’m trying to get this going:
http://ringzeroandlower.com/2017/08/08/x86-64-kernel-boot.html

The issue I’m running into is that nasm doesn’t seem to support 64-bit relocatable symbols in its elf-64 output, or at least I haven’t been able to figure it out. Basically after the kernel goes into 32-bit mode, it can start running code in virtual memory space, but that requires a 64-bit jump to go from the low 16/32 bit virtual addresses to the very large virtual memory address location. From what I can tell, x86-64 call instructions are actually always 64bit anyway, so it shouldn’t matter per se, but ld still requires that the relocation records explicitly be 64 bit.

nasm only seems to be able to write 32 bit relocation records:
RELOCATION RECORDS FOR [.boot_text]:
OFFSET TYPE VALUE
0000000000000010 R_X86_64_PC32 rust_main-0x0000000000000004

Though in the source code there does seem to be R_X86_64_PC64, it’s not obvious how to force it for a particular symbol. NASM’s docs seem pretty old, so it seems like the only thing to do is port my code to gcc.

Cool Article on Module Decomposition

We have tried to demonstrate by these examples that it is almost always incorrect to begin the decomposition of a system into modules on the basis of a flowchart. We propose instead that one begins with a list of difficult design decisions or design decisions which are likely to change. Each module is then designed to hide such a decision from the others. Since, in most cases, design decisions transcend time of execution, modules will not correspond to steps in the processing. To achieve an efficient implementation we must abandon the assumption that a module is one or more subroutines, and instead allow subroutines and programs to be assembled collections of code from various modules.

https://www.win.tue.nl/~wstomv/edu/2ip30/references/criteria_for_modularization.pdf

via https://hackernoon.com/you-dont-understand-the-single-responsibility-principle-abfdd005b137

Rust OS

I’ve been working on the Rust OS outlined in these posts:
https://os.phil-opp.com/advanced-paging/

They use a recursive page table map to map arbitrary pages to virtual memory. I’d like to use an offset mapping like the Linux kernel uses. In Linux, the kernel lives in a high memory area (above PAGE_OFFSET) and the user environments are mapped into the low memory area. Thinking about page tables makes my head spin though &em; it’s a real have to think 4th dimensionally kind of situation.

The issue is that on boot the pages are mapped in by the boot loader using its own scheme, and I’m not entirely sure what the mappings are. The kernel will have a bad time if it’s mapped away from where it is running, so if I do move to a different scheme, I’ll need to move the kernel with it. Alternatively, maybe I should modify the boot loader to map it in a way that is consistent with the final result.

Some questions I have:
1) what are the boot loader memory mappings?
2) if I move the kernel, could I just jump to the new location? are the addresses offsets or absolute?
3) could I copy the kernel pages to the equivalent offset pages? (are the equivalent pages guaranteed to be free?)

Some resources:
https://docs.rs/x86_64/0.3.5/x86_64/structures/paging/index.html
https://elinux.org/images/b/b0/Introduction_to_Memory_Management_in_Linux.pdf
https://www.kernel.org/doc/gorman/html/understand/understand006.html
https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-828-operating-system-engineering-fall-2012/labs/

Featured post queries

Get all posts that have featured images that aren’t attached to the post:
SELECT post.id, post.post_name FROM `wp_posts` post, wp_posts attach, wp_postmeta meta where meta.meta_key=’_thumbnail_id’ and meta.post_id=post.id and attach.id =
meta.meta_value and attach.post_parent != post.id

Use the first attached image as featured image:
INSERT INTO `wp_postmeta`
(`post_id`,
`meta_key`,
`meta_value`)
SELECT id,
‘_thumbnail_id’,
(SELECT id
FROM wp_posts b
WHERE a.id = b.post_parent
AND post_type = ‘attachment’
LIMIT 1) AS url
FROM `wp_posts` a
LEFT JOIN wp_postmeta
ON id = post_id
AND wp_postmeta.meta_key = ‘_thumbnail_id’
WHERE wp_postmeta.meta_id IS NULL
AND post_type = ‘post’
AND post_status = ‘publish’
AND post_date <= Subdate(Now(), INTERVAL 2 year) AND (SELECT guid FROM wp_posts b WHERE a.id = b.post_parent AND post_type = 'attachment' LIMIT 1) IS NOT NULL Move featured images not attached to featuring post: update wp_posts a, wp_posts p, wp_postmeta m set a.post_parent = p.id where p.id = m.post_id and meta_key='_thumbnail_id' and meta_value=a.id and p.id != a.post_parent;

The Grand Inquisitor, Dostoevsky

Freedom, free thought, and science will lead them into such straits and will bring them face to face with such marvels and insoluble mysteries, that some of them, the fierce and rebellious, will destroy themselves, others, rebellious but weak, will destroy one another, while the rest, weak and unhappy, will crawl fawning to our feet and whine to us: “Yes, you were right, you alone possess His mystery, and we come back to you, save us from ourselves!”

“‘Receiving bread from us, they will see clearly that we take the bread made by their hands from them, to give it to them, without any miracle. They will see that we do not change the stones to bread, but in truth they will be more thankful for taking it from our hands than for the bread itself! For they will remember only too well that in old days, without our help, even the bread they made turned to stones in their hands, while since they have come back to us, the very stones have turned to bread in their hands. Too, too well will they know the value of complete submission! And until men know that, they will be unhappy

using browser-based mocha with istanbul

Setting up mocha / chai / testdouble:


<div id="mocha"></div>
<div><a id="coverageLink">Export Coverage</a></div>

<script src="https://unpkg.com/[email protected]/chai.js"></script>
<script src="https://unpkg.com/[email protected]/mocha.js"></script>
<script src="https://unpkg.com/[email protected]/dist/testdouble.js"></script>
<script src="https://cdn.rawgit.com/jquery/jquery/2.1.4/dist/jquery.min.js"></script>

<script>mocha.setup({ ui: 'bdd',
globals: ['*'], ignoreLeaks: true
});</script>

<script src="./dist/bundle.js"></script>
<script src="./test/test.js"></script>

<script>
mocha.checkLeaks();
mocha.run();

document.getElementById('coverageLink').onclick = function() {
var json = JSON.stringify(window.__coverage__);
var jsonData = 'data:application/json;charset=utf-8,'
+ encodeURIComponent(json);
this.href = jsonData;
this.target = '_blank';
this.download = 'coverage.json';
};
</script>

Move coverage.json to coverage directory in src tree, then run:
yarn nyc report –temp-directory coverage/ -r lcov

Source: https://www.engineyard.com/blog/measuring-clientside-javascript-test-coverage-with-istanbul