Merge branch 'main' into update-mfm-core

CHANGELOG.md

@@ -11,6 +11,72 @@ Unreleased changes should not be listed in this file.
 Instead, run `git shortlog --format='%h %s' --group=trailer:changelog <last tag>..` to see unreleased changes; replace `<last tag>` with the tag you wish to compare from.
 If you are a contributor, please read [CONTRIBUTING.md, section "Changelog Trailer"](./CONTRIBUTING.md#changelog-trailer) on what to do instead.
 
+## 13.0.0-preview5 - 2023-05-23
+This release contains 6 breaking changes and 1 security update.
+
+### Security
+- client: check input for aiscript
+- server: validate filenames and emoji names on emoji import
+- server: check URL schema of ActivityPub URIs
+- server: check schema for URL previews
+- server: update summaly dependency
+
+### Added
+- client: impolement filtering and sorting in drive
+- client: add "nobody" follower/following visibility
+- client: re-add flag to require approval for bot follows
+- client: show waveform on audio player
+- client: add new deepl languages
+- client: add instructions on remote interaction (when signed out)
+- client: show follow button when not logged in
+- server: show worker mode in process names
+- server: drive endpoint to fetch files and folders combined
+- activitypub: implement receiving account moves
+
+### Changed
+- **BREAKING** server: restructure endpoints related to user administration
+- **BREAKING** server: refactor streaming API data structures
+- **BREAKING** server: rename configuration environment variables
+  The environment variables that could be used for configuration which were previously prefixed with `MK_`
+  are now prefixed with `FK_` instead.
+- server: improve error message for invalidating follows
+- server: add pagination to file attachment timeline
+
+### Fixed
+- **BREAKING** server: properly respect follower/following visibility setting on statistics endpoint
+  This affects the endpoint `/api/users/stats`.
+- improve documentation for `fetch-rss` endpoint
+- client: fix authentication error in RSS widget
+- client: fix attached files and account switcher combination in new note form
+- client: improved module tracker file detection
+- client: fix follow requests pagination
+- client: Theme creator breaks after creating a theme
+- client: replace error UUIDs with error codes
+- client: allow opening links in new tab
+  The usual 3rd button click (usually mouse wheel) or Ctrl+Click should now work to open a link in a new tab.
+- client: fix drive item updates inserting duplicate entries
+- client: improve error messages for failed uploads
+- client: stop unnecessary network congestion by websocket ping mechanism
+- server: don't fail if a system user was already created
+- server: better matching for MFM mentions
+- server: fix rate limit for adding reactions
+- server: check instance description length limit
+- server: dont error on generating RSS feeds for profiles without public posts
+- server: group delivering `Delete` activities to improve performance
+- server: fix drive quota for remote users
+- server: user deletion race condition (again)
+
+### Removed
+- **BREAKING** server: remove unused API parameters `sinceId` and `untilId` from `/api/notes/reactions`.
+- **BREAKING** server: remove syslog integration
+  If you used syslog before, the syslog protocoll will no longer be connected to.
+  The configuration entries for `syslog` will be ignored, you should remove them if they are set.
+- client: remove `driveFolderBg` theme colour
+- activitypub: remove `_misskey_content` attribute
+- activitypub: remove `_misskey_reaction` attribute
+- activitypub: remove `_misskey_votes` attribute
+- foundkey-js: remove unused definitions for Ads and detailed instance metadata
+
 ## 13.0.0-preview4 - 2023-02-05
 This release contains 6 breaking changes, including changes to the configuration file format.
 

docs/install-docker.md

@@ -38,10 +38,11 @@ cp .config/docker_example.env .config/docker.env
 Edit `default.yml` and `docker.env` according to the instructions in the files.
 You will need to set the database host to `db` and Redis host to `redis` in order to use the internal container network for these services.
 
-
 Edit `docker-compose.yml` if necessary. (e.g. if you want to change the port).
 If you are using SELinux (eg. you're on Fedora or a RHEL derivative), you'll want to add the `Z` mount flag to the volume mounts to allow the containers to access the contents of those volumes.
 
+Also check out the [Configure Foundkey](./install.md#configure-foundkey) section in the ordinary installation instructions.
+
 ## Build and initialize
 The following command will build FoundKey and initialize the database.
 This will take some time.

docs/install.md

@@ -78,6 +78,21 @@ There are instructions for setting up [nginx](./nginx.md) for this purpose.
 ### Changing the default Reaction
 You can change the default reaction that is used when an ActivityPub "Like" is received from '👍' to '⭐' by changing the boolean value `meta.useStarForReactionFallback` in the databse respectively.
 
+### Environment variables
+There are some behaviour changes which can be accomplished using environment variables.
+
+|variable name|meaning|
+|---|---|
+|`FK_ONLY_QUEUE`|If set, only the queue processing will be run. The frontend will not be available. Cannot be combined with `FK_ONLY_SERVER` or `FK_DISABLE_CLUSTERING`.|
+|`FK_ONLY_SERVER`|If set, only the frontend will be run. Queues will not be processed. Cannot be combined with `FK_ONLY_QUEUE` or `FK_DISABLE_CLUSTERING`.|
+|`FK_NO_DAEMONS`|If set, the server statistics and queue statistics will not be run.|
+|`FK_DISABLE_CLUSTERING`|If set, all work will be done in a single thread instead of different threads for frontend and queue. (not recommended)|
+|`FK_WITH_LOG_TIME`|If set, a timestamp will be appended to all log messages.|
+|`FK_SLOW`|If set, all requests will be delayed by 3s. (not recommended, useful for testing)|
+|`FK_LOG_LEVEL`|Sets the log level. Messages below the set log level will be suppressed. Available log levels are `quiet` (suppress all), `error`, `warning`, `success`, `info`, `debug`.|
+
+If the `NODE_ENV` environment variable is set to `testing`, then the flags `FK_DISABLE_CLUSTERING` and `FK_NO_DAEMONS` will always be set, and the log level will always be `quiet`.
+
 ## Build FoundKey
 
 Build foundkey with the following:

locales/en-US.yml

@@ -32,9 +32,6 @@ signup: "Sign Up"
 save: "Save"
 users: "Users"
 addUser: "Add a user"
-favorite: "Add to favorites"
-favorites: "Favorites"
-unfavorite: "Remove from favorites"
 pin: "Pin to profile"
 unpin: "Unpin from profile"
 copyContent: "Copy contents"

package.json

@@ -1,6 +1,6 @@
 {
 	"name": "foundkey",
-	"version": "13.0.0-preview4",
+	"version": "13.0.0-preview5",
 	"repository": {
 		"type": "git",
 		"url": "https://akkoma.dev/FoundKeyGang/FoundKey.git"

packages/backend/migration/1673201544000-deletion-progress.js

@@ -0,0 +1,18 @@
+export class deletionProgress1673201544000 {
+	name = 'deletionProgress1673201544000';
+
+	async up(queryRunner) {
+		await queryRunner.query(`ALTER TABLE "user" RENAME COLUMN "isDeleted" TO "isDeletedOld"`);
+		await queryRunner.query(`ALTER TABLE "user" ADD "isDeleted" integer`);
+		await queryRunner.query(`UPDATE "user" SET "isDeleted" = CASE WHEN "host" IS NULL THEN -1 ELSE 0 END WHERE "isDeletedOld"`);
+		await queryRunner.query(`ALTER TABLE "user" DROP COLUMN "isDeletedOld"`);
+	}
+
+	async down(queryRunner) {
+		await queryRunner.query(`ALTER TABLE "user" RENAME COLUMN "isDeleted" TO "isDeletedOld"`);
+		await queryRunner.query(`ALTER TABLE "user" ADD "isDeleted" boolean NOT NULL DEFAULT false`);
+		await queryRunner.query(`UPDATE "user" SET "isDeleted" = "isDeletedOld" IS NOT NULL`);
+		await queryRunner.query(`ALTER TABLE "user" DROP COLUMN "isDeletedOld"`);
+	}
+}
+

packages/backend/migration/1685126322423-remove-favourites.js

@@ -0,0 +1,33 @@
+export class removeFavourites1685126322423 {
+	name = 'removeFavourites1685126322423';
+
+	async up(queryRunner) {
+		await queryRunner.query(`
+			WITH "new_clips" AS (
+				INSERT INTO "clip" ("id", "createdAt", "userId", "name")
+				SELECT
+					RIGHT(GEN_RANDOM_UUID()::text, 10),
+					NOW(),
+					"userId",
+					'⭐'
+				FROM "note_favorite"
+				GROUP BY "userId"
+				RETURNING "id", "userId"
+			)
+			INSERT INTO "clip_note" ("id", "noteId", "clipId")
+			SELECT
+				"note_favorite"."id",
+				"noteId",
+				"new_clips"."id"
+			FROM "note_favorite"
+			JOIN "new_clips" ON "note_favorite"."userId" = "new_clips"."userId"
+		`);
+		await queryRunner.query(`DROP TABLE "note_favorite"`);
+	}
+
+	async down(queryRunner) {
+		// can't revert the migration to clips, can only recreate the database table
+		await queryRunner.query(`CREATE TABLE "note_favorite" ("id" character varying(32) NOT NULL, "createdAt" TIMESTAMP WITH TIME ZONE NOT NULL, "userId" character varying(32) NOT NULL, "noteId" character varying(32) NOT NULL, CONSTRAINT "PK_af0da35a60b9fa4463a62082b36" PRIMARY KEY ("id"))`);
+	}
+}
+

packages/backend/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "backend",
-	"version": "13.0.0-preview4",
+	"version": "13.0.0-preview5",
 	"main": "./index.js",
 	"private": true,
 	"type": "module",

packages/backend/src/boot/index.ts

@@ -3,7 +3,7 @@ import chalk from 'chalk';
 import Xev from 'xev';
 
 import Logger from '@/services/logger.js';
-import { envOption } from '@/env.js';
+import { envOption, LOG_LEVELS } from '@/env.js';
 
 // for typeorm
 import 'reflect-metadata';
@@ -66,7 +66,7 @@ cluster.on('exit', worker => {
 });
 
 // Display detail of unhandled promise rejection
-if (!envOption.quiet) {
+if (envOption.logLevel !== LOG_LEVELS.quiet) {
 	process.on('unhandledRejection', console.dir);
 }
 

packages/backend/src/boot/master.ts

@@ -11,7 +11,7 @@ import Logger from '@/services/logger.js';
 import { loadConfig } from '@/config/load.js';
 import { Config } from '@/config/types.js';
 import { showMachineInfo } from '@/misc/show-machine-info.js';
-import { envOption } from '@/env.js';
+import { envOption, LOG_LEVELS } from '@/env.js';
 import { db, initDb } from '@/db/postgre.js';
 
 const _filename = fileURLToPath(import.meta.url);
@@ -25,7 +25,7 @@ const bootLogger = logger.createSubLogger('boot', 'magenta', false);
 const themeColor = chalk.hex('#86b300');
 
 function greet(): void {
-	if (!envOption.quiet) {
+	if (envOption.logLevel !== LOG_LEVELS.quiet) {
 		//#region FoundKey logo
 		console.log(themeColor('  ___                 _ _  __         '));
 		console.log(themeColor(' | __|__ _  _ _ _  __| | |/ /___ _  _ '));
@@ -141,15 +141,24 @@ async function connectDb(): Promise<void> {
 
 async function spawnWorkers(clusterLimits: Required<Config['clusterLimits']>): Promise<void> {
 	const modes = ['web' as const, 'queue' as const];
+
+	const clusters = structuredClone(clusterLimits);
+
+	if (envOption.onlyQueue) {
+		clusters.web = 0;
+	} else if (envOption.onlyServer) {
+		clusters.queue = 0;
+	}
+
 	const cpus = os.cpus().length;
-	for (const mode of modes.filter(mode => clusterLimits[mode] > cpus)) {
+	for (const mode of modes.filter(mode => clusters[mode] > cpus)) {
 		bootLogger.warn(`configuration warning: cluster limit for ${mode} exceeds number of cores (${cpus})`);
 	}
 
-	const total = modes.reduce((acc, mode) => acc + clusterLimits[mode], 0);
+	const total = modes.reduce((acc, mode) => acc + clusters[mode], 0);
 	const workers = new Array(total);
-	workers.fill('web', 0, clusterLimits.web);
-	workers.fill('queue', clusterLimits.web);
+	workers.fill('web', 0, clusters.web);
+	workers.fill('queue', clusters.web);
 
 	bootLogger.info(`Starting ${total} workers...`);
 	await Promise.all(workers.map(mode => spawnWorker(mode)));

packages/backend/src/db/postgre.ts

@@ -33,7 +33,6 @@ import { UserGroup } from '@/models/entities/user-group.js';
 import { UserGroupJoining } from '@/models/entities/user-group-joining.js';
 import { UserGroupInvitation } from '@/models/entities/user-group-invitation.js';
 import { Hashtag } from '@/models/entities/hashtag.js';
-import { NoteFavorite } from '@/models/entities/note-favorite.js';
 import { AbuseUserReport } from '@/models/entities/abuse-user-report.js';
 import { RegistrationTicket } from '@/models/entities/registration-tickets.js';
 import { MessagingMessage } from '@/models/entities/messaging-message.js';
@@ -134,7 +133,6 @@ export const entities = [
 	RenoteMuting,
 	Blocking,
 	Note,
-	NoteFavorite,
 	NoteReaction,
 	NoteWatching,
 	NoteThreadMuting,

packages/backend/src/env.ts

@@ -1,20 +1,38 @@
-const envOption = {
+export const LOG_LEVELS = {
+	quiet: 6,
+	error: 5,
+	warning: 4,
+	success: 3,
+	info: 2,
+	debug: 1,
+};
+
+export const envOption = {
 	onlyQueue: false,
 	onlyServer: false,
 	noDaemons: false,
 	disableClustering: false,
-	verbose: false,
 	withLogTime: false,
-	quiet: false,
 	slow: false,
+	logLevel: LOG_LEVELS.info,
 };
 
 for (const key of Object.keys(envOption) as (keyof typeof envOption)[]) {
-	if (process.env['MK_' + key.replace(/[A-Z]/g, letter => `_${letter}`).toUpperCase()]) envOption[key] = true;
+	const value = process.env['FK_' + key.replace(/[A-Z]/g, letter => `_${letter}`).toUpperCase()];
+	if (value) {
+		if (key === 'logLevel') {
+			if (value.toLowerCase() in LOG_LEVELS) {
+				envOption.logLevel = LOG_LEVELS[value.toLowerCase()];
+			}
+			console.log('Unknown log level ' + JSON.stringify(value.toLowerCase()) + ', defaulting to "info"');
+		} else {
+			envOption[key] = true;
+		}
+	}
 }
 
-if (process.env.NODE_ENV === 'test') envOption.disableClustering = true;
-if (process.env.NODE_ENV === 'test') envOption.quiet = true;
-if (process.env.NODE_ENV === 'test') envOption.noDaemons = true;
-
-export { envOption };
+if (process.env.NODE_ENV === 'test') {
+	envOption.disableClustering = true;
+	envOption.logLevel = LOG_LEVELS.quiet;
+	envOption.noDaemons = true;
+}

packages/backend/src/misc/api-permissions.ts

@@ -5,8 +5,6 @@ export const kinds = [
 	'write:blocks',
 	'read:drive',
 	'write:drive',
-	'read:favorites',
-	'write:favorites',
 	'read:following',
 	'write:following',
 	'read:messaging',

packages/backend/src/misc/schema.ts

@@ -23,7 +23,6 @@ import { packedHashtagSchema } from '@/models/schema/hashtag.js';
 import { packedPageSchema } from '@/models/schema/page.js';
 import { packedUserGroupSchema } from '@/models/schema/user-group.js';
 import { packedUserGroupInvitationSchema } from '@/models/schema/user-group-invitation.js';
-import { packedNoteFavoriteSchema } from '@/models/schema/note-favorite.js';
 import { packedChannelSchema } from '@/models/schema/channel.js';
 import { packedAntennaSchema } from '@/models/schema/antenna.js';
 import { packedClipSchema } from '@/models/schema/clip.js';
@@ -47,7 +46,6 @@ export const refs = {
 	MessagingMessage: packedMessagingMessageSchema,
 	Note: packedNoteSchema,
 	NoteReaction: packedNoteReactionSchema,
-	NoteFavorite: packedNoteFavoriteSchema,
 	Notification: packedNotificationSchema,
 	DriveFile: packedDriveFileSchema,
 	DriveFolder: packedDriveFolderSchema,

packages/backend/src/models/entities/note-favorite.ts

@@ -1,35 +0,0 @@
-import { PrimaryColumn, Entity, Index, JoinColumn, Column, ManyToOne } from 'typeorm';
-import { id } from '../id.js';
-import { Note } from './note.js';
-import { User } from './user.js';
-
-@Entity()
-@Index(['userId', 'noteId'], { unique: true })
-export class NoteFavorite {
-	@PrimaryColumn(id())
-	public id: string;
-
-	@Column('timestamp with time zone', {
-		comment: 'The created date of the NoteFavorite.',
-	})
-	public createdAt: Date;
-
-	@Index()
-	@Column(id())
-	public userId: User['id'];
-
-	@ManyToOne(() => User, {
-		onDelete: 'CASCADE',
-	})
-	@JoinColumn()
-	public user: User | null;
-
-	@Column(id())
-	public noteId: Note['id'];
-
-	@ManyToOne(() => Note, {
-		onDelete: 'CASCADE',
-	})
-	@JoinColumn()
-	public note: Note | null;
-}

packages/backend/src/models/entities/user.ts

@@ -163,11 +163,11 @@ export class User {
 	// Indicates the user was deleted by an admin.
 	// The users' data is not deleted from the database to keep them from reappearing.
 	// A hard delete of the record may follow if we receive a matching Delete activity.
-	@Column('boolean', {
-		default: false,
-		comment: 'Whether the User is deleted.',
+	@Column('integer', {
+		nullable: true,
+		comment: 'How many delivery jobs are outstanding before the deletion is completed.',
 	})
-	public isDeleted: boolean;
+	public isDeleted: number | null;
 
 	@Column('varchar', {
 		length: 128, array: true, default: '{}',

packages/backend/src/models/index.ts

@@ -29,7 +29,6 @@ import { RenoteMutingRepository } from './repositories/renote-muting.js';
 import { BlockingRepository } from './repositories/blocking.js';
 import { NoteReactionRepository } from './repositories/note-reaction.js';
 import { NotificationRepository } from './repositories/notification.js';
-import { NoteFavoriteRepository } from './repositories/note-favorite.js';
 import { UserPublickey } from './entities/user-publickey.js';
 import { UserKeypair } from './entities/user-keypair.js';
 import { AppRepository } from './repositories/app.js';
@@ -64,7 +63,6 @@ export const Announcements = db.getRepository(Announcement);
 export const AnnouncementReads = db.getRepository(AnnouncementRead);
 export const Apps = (AppRepository);
 export const Notes = (NoteRepository);
-export const NoteFavorites = (NoteFavoriteRepository);
 export const NoteWatchings = db.getRepository(NoteWatching);
 export const NoteThreadMutings = db.getRepository(NoteThreadMuting);
 export const NoteReactions = (NoteReactionRepository);

packages/backend/src/models/repositories/note-favorite.ts

@@ -1,29 +0,0 @@
-import { db } from '@/db/postgre.js';
-import { NoteFavorite } from '@/models/entities/note-favorite.js';
-import { User } from '@/models/entities/user.js';
-import { Notes } from '../index.js';
-
-export const NoteFavoriteRepository = db.getRepository(NoteFavorite).extend({
-	async pack(
-		src: NoteFavorite['id'] | NoteFavorite,
-		me?: { id: User['id'] } | null | undefined,
-	) {
-		const favorite = typeof src === 'object' ? src : await this.findOneByOrFail({ id: src });
-
-		return {
-			id: favorite.id,
-			createdAt: favorite.createdAt.toISOString(),
-			noteId: favorite.noteId,
-			// may throw error
-			note: await Notes.pack(favorite.note || favorite.noteId, me),
-		};
-	},
-
-	packMany(
-		favorites: any[],
-		me: { id: User['id'] },
-	) {
-		return Promise.allSettled(favorites.map(x => this.pack(x, me)))
-		.then(promises => promises.flatMap(result => result.status === 'fulfilled' ? [result.value] : []));
-	},
-});

packages/backend/src/models/repositories/user.ts

@@ -258,7 +258,7 @@ export const UserRepository = db.getRepository(User).extend({
 			case 'nobody':
 				return false;
 		}
-	}
+	},
 
 	async pack<ExpectsMe extends boolean | null = null, D extends boolean = false>(
 		src: User['id'] | User,
@@ -381,7 +381,7 @@ export const UserRepository = db.getRepository(User).extend({
 				autoAcceptFollowed: profile!.autoAcceptFollowed,
 				noCrawle: profile!.noCrawle,
 				isExplorable: user.isExplorable,
-				isDeleted: user.isDeleted,
+				isDeleted: user.isDeleted != null,
 				hideOnlineStatus: user.hideOnlineStatus,
 				hasUnreadSpecifiedNotes: NoteUnreads.count({
 					where: { userId: user.id, isSpecified: true },

packages/backend/src/models/schema/note-favorite.ts

@@ -1,26 +0,0 @@
-export const packedNoteFavoriteSchema = {
-	type: 'object',
-	properties: {
-		id: {
-			type: 'string',
-			optional: false, nullable: false,
-			format: 'id',
-			example: 'xxxxxxxxxx',
-		},
-		createdAt: {
-			type: 'string',
-			optional: false, nullable: false,
-			format: 'date-time',
-		},
-		note: {
-			type: 'object',
-			optional: false, nullable: false,
-			ref: 'Note',
-		},
-		noteId: {
-			type: 'string',
-			optional: false, nullable: false,
-			format: 'id',
-		},
-	},
-} as const;

packages/backend/src/queue/index.ts

@@ -1,11 +1,12 @@
 import httpSignature from '@peertube/http-signature';
 import { v4 as uuid } from 'uuid';
+import Bull from 'bull';
 
 import config from '@/config/index.js';
+import { Users } from '@/models/index.js';
 import { DriveFile } from '@/models/entities/drive-file.js';
 import { Webhook, webhookEventTypes } from '@/models/entities/webhook.js';
 import { IActivity } from '@/remote/activitypub/type.js';
-import { envOption } from '@/env.js';
 import { MINUTE } from '@/const.js';
 
 import processDeliver from './processors/deliver.js';
@@ -18,7 +19,7 @@ import { endedPollNotification } from './processors/ended-poll-notification.js';
 import { queueLogger } from './logger.js';
 import { getJobInfo } from './get-job-info.js';
 import { systemQueue, dbQueue, deliverQueue, inboxQueue, objectStorageQueue, endedPollNotificationQueue, webhookDeliverQueue } from './queues.js';
-import { ThinUser } from './types.js';
+import { DeliverJobData, ThinUser } from './types.js';
 
 function renderError(e: Error): any {
 	return {
@@ -35,6 +36,12 @@ const inboxLogger = queueLogger.createSubLogger('inbox');
 const dbLogger = queueLogger.createSubLogger('db');
 const objectStorageLogger = queueLogger.createSubLogger('objectStorage');
 
+async function deletionRefCount(job: Bull.Job<DeliverJobData>): Promise<void> {
+	if (job.data.deletingUserId) {
+		await Users.decrement({ id: job.data.deletingUserId }, 'isDeleted', 1);
+	}
+}
+
 systemQueue
 	.on('waiting', (jobId) => systemLogger.debug(`waiting id=${jobId}`))
 	.on('active', (job) => systemLogger.debug(`active id=${job.id}`))
@@ -46,8 +53,14 @@ systemQueue
 deliverQueue
 	.on('waiting', (jobId) => deliverLogger.debug(`waiting id=${jobId}`))
 	.on('active', (job) => deliverLogger.debug(`active ${getJobInfo(job, true)} to=${job.data.to}`))
-	.on('completed', (job, result) => deliverLogger.debug(`completed(${result}) ${getJobInfo(job, true)} to=${job.data.to}`))
-	.on('failed', (job, err) => deliverLogger.warn(`failed(${err}) ${getJobInfo(job)} to=${job.data.to}`))
+	.on('completed', async (job, result) => {
+		deliverLogger.debug(`completed(${result}) ${getJobInfo(job, true)} to=${job.data.to}`);
+		await deletionRefCount(job);
+	})
+	.on('failed', async (job, err) => {
+		deliverLogger.warn(`failed(${err}) ${getJobInfo(job)} to=${job.data.to}`);
+		await deletionRefCount(job);
+	})
 	.on('error', (job: any, err: Error) => deliverLogger.error(`error ${err}`))
 	.on('stalled', (job) => deliverLogger.warn(`stalled ${getJobInfo(job)} to=${job.data.to}`));
 
@@ -83,7 +96,7 @@ webhookDeliverQueue
 	.on('error', (job: any, err: Error) => webhookLogger.error(`error ${err}`))
 	.on('stalled', (job) => webhookLogger.warn(`stalled ${getJobInfo(job)} to=${job.data.to}`));
 
-export function deliver(user: ThinUser, content: unknown, to: string | null) {
+export function deliver(user: ThinUser, content: unknown, to: string | null, deletingUserId?: string) {
 	if (content == null) return null;
 	if (to == null) return null;
 
@@ -93,6 +106,7 @@ export function deliver(user: ThinUser, content: unknown, to: string | null) {
 		},
 		content,
 		to,
+		deletingUserId,
 	};
 
 	return deliverQueue.add(data, {
@@ -289,8 +303,6 @@ export function webhookDeliver(webhook: Webhook, type: typeof webhookEventTypes[
 }
 
 export default function() {
-	if (envOption.onlyServer) return;
-
 	deliverQueue.process(config.deliverJobConcurrency, processDeliver);
 	inboxQueue.process(config.inboxJobConcurrency, processInbox);
 	endedPollNotificationQueue.process(endedPollNotification);
@@ -326,8 +338,9 @@ export default function() {
 }
 
 export function destroy() {
-	deliverQueue.once('cleaned', (jobs, status) => {
+	deliverQueue.once('cleaned', async (jobs, status) => {
 		deliverLogger.succ(`Cleaned ${jobs.length} ${status} jobs`);
+		await Promise.all(jobs.map(job => deletionRefCount(job)));
 	});
 	deliverQueue.clean(0, 'delayed');
 

packages/backend/src/queue/processors/db/delete-account.ts

@@ -46,29 +46,17 @@ export async function deleteAccount(job: Bull.Job<DbUserDeleteJobData>): Promise
 	}
 
 	{ // Delete files
-		let cursor: DriveFile['id'] | null = null;
+		const files = await DriveFiles.find({
+			where: {
+				userId: user.id,
+			},
+			order: {
+				id: 1,
+			},
+		}) as DriveFile[];
 
-		while (true) {
-			const files = await DriveFiles.find({
-				where: {
-					userId: user.id,
-					...(cursor ? { id: MoreThan(cursor) } : {}),
-				},
-				take: 10,
-				order: {
-					id: 1,
-				},
-			}) as DriveFile[];
-
-			if (files.length === 0) {
-				break;
-			}
-
-			cursor = files[files.length - 1].id;
-
-			for (const file of files) {
-				await deleteFileSync(file);
-			}
+		for (const file of files) {
+			await deleteFileSync(file);
 		}
 
 		logger.succ('All of files deleted');

packages/backend/src/queue/processors/system/check-expired.ts

@@ -1,6 +1,6 @@
 import Bull from 'bull';
 import { In, LessThan } from 'typeorm';
-import { AttestationChallenges, AuthSessions, Mutings, Notifications, PasswordResetRequests, Signins } from '@/models/index.js';
+import { AttestationChallenges, AuthSessions, Mutings, Notifications, PasswordResetRequests, Signins, Users } from '@/models/index.js';
 import { publishUserEvent } from '@/services/stream.js';
 import { MINUTE, MONTH } from '@/const.js';
 import { queueLogger } from '@/queue/logger.js';
@@ -52,6 +52,11 @@ export async function checkExpired(job: Bull.Job<Record<string, unknown>>, done:
 		createdAt: OlderThan(3 * MONTH),
 	});
 
+	await Users.delete({
+		// delete users where the deletion status reference count has come down to zero
+		isDeleted: 0,
+	});
+
 	logger.succ('Deleted expired data.');
 
 	done();

packages/backend/src/queue/types.ts

@@ -12,6 +12,8 @@ export type DeliverJobData = {
 	content: unknown;
 	/** inbox URL to deliver */
 	to: string;
+	/** set if this job is part of a user deletion, on completion or failure the isDeleted field needs to be decremented */
+	deletingUserId?: string;
 };
 
 export type InboxJobData = {

packages/backend/src/remote/activitypub/deliver-manager.ts

@@ -88,10 +88,10 @@ export class DeliverManager {
 	/**
 	 * Execute delivers
 	 */
-	public async execute() {
+	public async execute(deletingUserId?: string) {
 		if (!Users.isLocalUser(this.actor)) return;
 
-		const inboxes = new Set<string>();
+		let inboxes = new Set<string>();
 
 		/*
 		build inbox list
@@ -150,13 +150,17 @@ export class DeliverManager {
 			)),
 		);
 
-		// deliver
-		for (const inbox of inboxes) {
-			// skip instances as indicated
-			if (instancesToSkip.includes(new URL(inbox).host)) continue;
+		const filteredInboxes = Array.from(inboxes)
+			.filter(inbox => !instancesToSkip.includes(new URL(inbox).host));
 
-			deliver(this.actor, this.activity, inbox);
+		if (deletingUserId) {
+			await Users.update(deletingUserId, {
+				// set deletion job count for reference counting before queueing jobs
+				isDeleted: filteredInboxes.length,
+			});
 		}
+
+		filteredInboxes.forEach(inbox => deliver(this.actor, this.activity, inbox, deletingUserId));
 	}
 }
 

packages/backend/src/remote/activitypub/kernel/delete/actor.ts

@@ -16,7 +16,7 @@ export async function deleteActor(actor: IRemoteUser, uri: string): Promise<stri
 		// anyway, the user is gone now so dont care
 		return 'ok: gone';
 	}
-	if (user.isDeleted) {
+	if (user.isDeleted != null) {
 		// the actual deletion already happened by an admin, just delete the record
 		await Users.delete(actor.id);
 	} else {

packages/backend/src/server/api/api-handler.ts

@@ -45,7 +45,10 @@ export async function handler(endpoint: IEndpoint, ctx: Koa.Context): Promise<vo
 		if (e instanceof AuthenticationError) {
 			new ApiError('AUTHENTICATION_FAILED', e.message).apply(ctx, endpoint.name);
 		} else {
-			new ApiError().apply(ctx, endpoint.name);
+			new ApiError('INTERNAL_ERROR', {
+				e: e.message,
+				stack: e.stack,
+			}).apply(ctx, endpoint.name);
 		}
 	});
 }

packages/backend/src/server/api/common/getters.ts

@@ -32,7 +32,7 @@ export async function getNote(noteId: Note['id'], me: { id: User['id'] } | null)
 export async function getUser(userId: User['id'], includeSuspended = false) {
 	const user = await Users.findOneBy({
 		id: userId,
-		isDeleted: false,
+		isDeleted: IsNull(),
 		...(includeSuspended ? {} : {isSuspended: false}),
 	});
 
@@ -50,7 +50,7 @@ export async function getRemoteUser(userId: User['id'], includeSuspended = false
 	const user = await Users.findOneBy({
 		id: userId,
 		host: Not(IsNull()),
-		isDeleted: false,
+		isDeleted: IsNull(),
 		...(includeSuspended ? {} : {isSuspended: false}),
 	});
 
@@ -68,7 +68,7 @@ export async function getLocalUser(userId: User['id'], includeSuspended = false)
 	const user = await Users.findOneBy({
 		id: userId,
 		host: IsNull(),
-		isDeleted: false,
+		isDeleted: IsNull(),
 		...(includeSuspended ? {} : {isSuspended: false}),
 	});
 

packages/backend/src/server/api/endpoints.ts

@@ -162,7 +162,6 @@ import * as ep___i_exportFollowing from './endpoints/i/export-following.js';
 import * as ep___i_exportMute from './endpoints/i/export-mute.js';
 import * as ep___i_exportNotes from './endpoints/i/export-notes.js';
 import * as ep___i_exportUserLists from './endpoints/i/export-user-lists.js';
-import * as ep___i_favorites from './endpoints/i/favorites.js';
 import * as ep___i_getWordMutedNotesCount from './endpoints/i/get-word-muted-notes-count.js';
 import * as ep___i_importBlocking from './endpoints/i/import-blocking.js';
 import * as ep___i_importFollowing from './endpoints/i/import-following.js';
@@ -215,8 +214,6 @@ import * as ep___notes_clips from './endpoints/notes/clips.js';
 import * as ep___notes_conversation from './endpoints/notes/conversation.js';
 import * as ep___notes_create from './endpoints/notes/create.js';
 import * as ep___notes_delete from './endpoints/notes/delete.js';
-import * as ep___notes_favorites_create from './endpoints/notes/favorites/create.js';
-import * as ep___notes_favorites_delete from './endpoints/notes/favorites/delete.js';
 import * as ep___notes_featured from './endpoints/notes/featured.js';
 import * as ep___notes_globalTimeline from './endpoints/notes/global-timeline.js';
 import * as ep___notes_hybridTimeline from './endpoints/notes/hybrid-timeline.js';
@@ -459,7 +456,6 @@ const eps = [
 	['i/export-mute', ep___i_exportMute],
 	['i/export-notes', ep___i_exportNotes],
 	['i/export-user-lists', ep___i_exportUserLists],
-	['i/favorites', ep___i_favorites],
 	['i/get-word-muted-notes-count', ep___i_getWordMutedNotesCount],
 	['i/import-blocking', ep___i_importBlocking],
 	['i/import-following', ep___i_importFollowing],
@@ -512,8 +508,6 @@ const eps = [
 	['notes/conversation', ep___notes_conversation],
 	['notes/create', ep___notes_create],
 	['notes/delete', ep___notes_delete],
-	['notes/favorites/create', ep___notes_favorites_create],
-	['notes/favorites/delete', ep___notes_favorites_delete],
 	['notes/featured', ep___notes_featured],
 	['notes/global-timeline', ep___notes_globalTimeline],
 	['notes/hybrid-timeline', ep___notes_hybridTimeline],

packages/backend/src/server/api/endpoints/admin/users/delete.ts

@@ -1,3 +1,4 @@
+import { IsNull } from 'typeorm';
 import { Users } from '@/models/index.js';
 import { ApiError } from '@/server/api/error.js';
 import { deleteAccount } from '@/services/delete-account.js';

packages/backend/src/server/api/endpoints/i/delete-account.ts

@@ -27,7 +27,7 @@ export default define(meta, paramDef, async (ps, user) => {
 		Users.findOneByOrFail({ id: user.id }),
 	]);
 
-	if (userDetailed.isDeleted) {
+	if (userDetailed.isDeleted != null) {
 		return;
 	}
 

packages/backend/src/server/api/endpoints/i/favorites.ts

@@ -1,44 +0,0 @@
-import { NoteFavorites } from '@/models/index.js';
-import define from '@/server/api/define.js';
-import { makePaginationQuery } from '@/server/api/common/make-pagination-query.js';
-
-export const meta = {
-	tags: ['account', 'notes', 'favorites'],
-
-	requireCredential: true,
-
-	kind: 'read:favorites',
-
-	res: {
-		type: 'array',
-		optional: false, nullable: false,
-		items: {
-			type: 'object',
-			optional: false, nullable: false,
-			ref: 'NoteFavorite',
-		},
-	},
-} as const;
-
-export const paramDef = {
-	type: 'object',
-	properties: {
-		limit: { type: 'integer', minimum: 1, maximum: 100, default: 10 },
-		sinceId: { type: 'string', format: 'misskey:id' },
-		untilId: { type: 'string', format: 'misskey:id' },
-	},
-	required: [],
-} as const;
-
-// eslint-disable-next-line import/no-default-export
-export default define(meta, paramDef, async (ps, user) => {
-	const query = makePaginationQuery(NoteFavorites.createQueryBuilder('favorite'), ps.sinceId, ps.untilId)
-		.andWhere('favorite.userId = :meId', { meId: user.id })
-		.leftJoinAndSelect('favorite.note', 'note');
-
-	const favorites = await query
-		.take(ps.limit)
-		.getMany();
-
-	return await NoteFavorites.packMany(favorites, user);
-});

packages/backend/src/server/api/endpoints/notes/favorites/create.ts

@@ -1,48 +0,0 @@
-import { NoteFavorites } from '@/models/index.js';
-import { genId } from '@/misc/gen-id.js';
-import define from '@/server/api/define.js';
-import { ApiError } from '@/server/api/error.js';
-import { getNote } from '@/server/api/common/getters.js';
-
-export const meta = {
-	tags: ['notes', 'favorites'],
-
-	requireCredential: true,
-
-	kind: 'write:favorites',
-
-	errors: ['NO_SUCH_NOTE', 'ALREADY_FAVORITED'],
-} as const;
-
-export const paramDef = {
-	type: 'object',
-	properties: {
-		noteId: { type: 'string', format: 'misskey:id' },
-	},
-	required: ['noteId'],
-} as const;
-
-// eslint-disable-next-line import/no-default-export
-export default define(meta, paramDef, async (ps, user) => {
-	// Get favoritee
-	const note = await getNote(ps.noteId, user).catch(err => {
-		if (err.id === '9725d0ce-ba28-4dde-95a7-2cbb2c15de24') throw new ApiError('NO_SUCH_NOTE');
-		throw err;
-	});
-
-	// if already favorited
-	const exist = await NoteFavorites.countBy({
-		noteId: note.id,
-		userId: user.id,
-	});
-
-	if (exist) throw new ApiError('ALREADY_FAVORITED');
-
-	// Create favorite
-	await NoteFavorites.insert({
-		id: genId(),
-		createdAt: new Date(),
-		noteId: note.id,
-		userId: user.id,
-	});
-});

packages/backend/src/server/api/endpoints/notes/favorites/delete.ts

@@ -1,42 +0,0 @@
-import { NoteFavorites } from '@/models/index.js';
-import define from '@/server/api/define.js';
-import { ApiError } from '@/server/api/error.js';
-import { getNote } from '@/server/api/common/getters.js';
-
-export const meta = {
-	tags: ['notes', 'favorites'],
-
-	requireCredential: true,
-
-	kind: 'write:favorites',
-
-	errors: ['NO_SUCH_NOTE', 'NOT_FAVORITED'],
-} as const;
-
-export const paramDef = {
-	type: 'object',
-	properties: {
-		noteId: { type: 'string', format: 'misskey:id' },
-	},
-	required: ['noteId'],
-} as const;
-
-// eslint-disable-next-line import/no-default-export
-export default define(meta, paramDef, async (ps, user) => {
-	// Get favoritee
-	const note = await getNote(ps.noteId, user).catch(err => {
-		if (err.id === '9725d0ce-ba28-4dde-95a7-2cbb2c15de24') throw new ApiError('NO_SUCH_NOTE');
-		throw err;
-	});
-
-	// if already favorited
-	const exist = await NoteFavorites.findOneBy({
-		noteId: note.id,
-		userId: user.id,
-	});
-
-	if (exist == null) throw new ApiError('NOT_FAVORITED');
-
-	// Delete favorite
-	await NoteFavorites.delete(exist.id);
-});

packages/backend/src/server/api/endpoints/notes/state.ts

@@ -1,4 +1,4 @@
-import { NoteFavorites, NoteThreadMutings, NoteWatchings } from '@/models/index.js';
+import { NoteThreadMutings, NoteWatchings } from '@/models/index.js';
 import { ApiError } from '@/server/api/error.js';
 import { getNote } from '@/server/api/common/getters.js';
 import define from '@/server/api/define.js';
@@ -12,10 +12,6 @@ export const meta = {
 		type: 'object',
 		optional: false, nullable: false,
 		properties: {
-			isFavorited: {
-				type: 'boolean',
-				optional: false, nullable: false,
-			},
 			isWatching: {
 				type: 'boolean',
 				optional: false, nullable: false,
@@ -51,14 +47,7 @@ export default define(meta, paramDef, async (ps, user) => {
 		throw err;
 	});
 
-	const [favorite, watching, threadMuting] = await Promise.all([
-		NoteFavorites.count({
-			where: {
-				userId: user.id,
-				noteId: note.id,
-			},
-			take: 1,
-		}),
+	const [watching, threadMuting] = await Promise.all([
 		NoteWatchings.count({
 			where: {
 				userId: user.id,
@@ -76,7 +65,6 @@ export default define(meta, paramDef, async (ps, user) => {
 	]);
 
 	return {
-		isFavorited: favorite !== 0,
 		isWatching: watching !== 0,
 		isMutedThread: threadMuting !== 0,
 	};

packages/backend/src/server/api/endpoints/users/stats.ts

@@ -1,4 +1,4 @@
-import { DriveFiles, Followings, NoteFavorites, NoteReactions, Notes, PageLikes, PollVotes, Users } from '@/models/index.js';
+import { DriveFiles, Followings, NoteReactions, Notes, PageLikes, PollVotes, Users } from '@/models/index.js';
 import { awaitAll } from '@/prelude/await-all.js';
 import define from '@/server/api/define.js';
 import { ApiError } from '@/server/api/error.js';
@@ -76,10 +76,6 @@ export const meta = {
 				type: 'integer',
 				optional: false, nullable: false,
 			},
-			noteFavoritesCount: {
-				type: 'integer',
-				optional: false, nullable: false,
-			},
 			pageLikesCount: {
 				type: 'integer',
 				optional: false, nullable: false,
@@ -148,9 +144,6 @@ export default define(meta, paramDef, async (ps, me) => {
 			.innerJoin('reaction.note', 'note')
 			.where('note.userId = :userId', { userId: user.id })
 			.getCount(),
-		noteFavoritesCount: NoteFavorites.createQueryBuilder('favorite')
-			.where('favorite.userId = :userId', { userId: user.id })
-			.getCount(),
 		pageLikesCount: PageLikes.createQueryBuilder('like')
 			.where('like.userId = :userId', { userId: user.id })
 			.getCount(),

packages/backend/src/server/api/error.ts

@@ -68,10 +68,6 @@ export const errors: Record<string, { message: string, httpStatusCode: number }>
 		message: 'That note is already added to that clip.',
 		httpStatusCode: 409,
 	},
-	ALREADY_FAVORITED: {
-		message: 'That note is already favorited.',
-		httpStatusCode: 409,
-	},
 	ALREADY_FOLLOWING: {
 		message: 'You are already following that user.',
 		httpStatusCode: 409,
@@ -332,10 +328,6 @@ export const errors: Record<string, { message: string, httpStatusCode: number }>
 		message: 'That note is not added to that clip.',
 		httpStatusCode: 409,
 	},
-	NOT_FAVORITED: {
-		message: 'You have not favorited that note.',
-		httpStatusCode: 409,
-	},
 	NOT_FOLLOWING: {
 		message: 'You are not following that user.',
 		httpStatusCode: 409,

packages/backend/src/server/api/private/signin.ts

@@ -47,7 +47,7 @@ export default async (ctx: Koa.Context) => {
 	const user = await Users.findOneBy({
 		usernameLower: username.toLowerCase(),
 		host: IsNull(),
-		isDeleted: false,
+		isDeleted: IsNull(),
 	}) as ILocalUser;
 
 	if (user == null) {

packages/backend/src/server/api/streaming.ts

@@ -44,18 +44,21 @@ export const initializeStreamingServer = (server: http.Server): void => {
 			const main = new Connection(socket, ev, user, app);
 
 			// ping/pong mechanism
-			let pingTimeout = null;
-			function startHeartbeat() {
-				if (pingTimeout) clearTimeout(pingTimeout);
-
+			let pingTimeout: NodeJS.Timeout | null = null;
+			let disconnectTimeout = setTimeout(() => {
+				socket.terminate();
+			}, 60 * SECOND);;
+			function sendPing() {
 				socket.ping();
 				pingTimeout = setTimeout(() => {
-					socket.terminate();
+					sendPing();
 				}, 30 * SECOND);
 			}
-			startHeartbeat();
-			socket.on('ping', () => { startHeartbeat(); });
-			socket.on('pong', () => { startHeartbeat(); });
+			function onPong() {
+				disconnectTimeout.refresh()
+			}
+			sendPing();
+			socket.on('pong', onPong);
 
 			// keep user "online" while a stream is connected
 			const intervalId = user ? setInterval(() => {
@@ -75,6 +78,7 @@ export const initializeStreamingServer = (server: http.Server): void => {
 				redisClient.off('message', onRedisMessage);
 				if (intervalId) clearInterval(intervalId);
 				if (pingTimeout) clearTimeout(pingTimeout);
+				if (disconnectTimeout) clearTimeout(disconnectTimeout);
 			});
 		});
 	});

packages/backend/src/server/web/index.ts

@@ -223,7 +223,7 @@ const getFeed = async (acct: string) => {
 		usernameLower: username.toLowerCase(),
 		host: host ?? IsNull(),
 		isSuspended: false,
-		isDeleted: false,
+		isDeleted: IsNull(),
 	});
 
 	return user && await packFeed(user);
@@ -273,7 +273,7 @@ router.get(['/@:user', '/@:user/:sub'], async (ctx, next) => {
 		usernameLower: username.toLowerCase(),
 		host: host ?? IsNull(),
 		isSuspended: false,
-		isDeleted: false,
+		isDeleted: IsNull(),
 	});
 
 	if (user != null) {
@@ -306,7 +306,7 @@ router.get('/users/:user', async ctx => {
 		id: ctx.params.user,
 		host: IsNull(),
 		isSuspended: false,
-		isDeleted: false,
+		isDeleted: IsNull(),
 	});
 
 	if (user == null) {
@@ -423,7 +423,7 @@ router.get('/@:user/pages/:page', async (ctx, next) => {
 		usernameLower: username.toLowerCase(),
 		host: host ?? IsNull(),
 		isSuspended: false,
-		isDeleted: false,
+		isDeleted: IsNull(),
 	});
 
 	if (user == null) return;

packages/backend/src/services/delete-account.ts

@@ -9,7 +9,7 @@ export async function deleteAccount(user: {
 }): Promise<void> {
 	await Promise.all([
 		Users.update(user.id, {
-			isDeleted: true,
+			isDeleted: -1,
 		}),
 		// revoke all of the users access tokens to block API access
 		AccessTokens.delete({

packages/backend/src/services/drive/add-file.ts

@@ -2,9 +2,10 @@ import * as fs from 'node:fs';
 
 import { v4 as uuid } from 'uuid';
 import S3 from 'aws-sdk/clients/s3.js';
-import { IsNull } from 'typeorm';
+import { In, IsNull } from 'typeorm';
 import sharp from 'sharp';
 
+import { db } from '@/db/postgre.js';
 import { FILE_TYPE_BROWSERSAFE } from '@/const.js';
 import { publishMainStream, publishDriveStream } from '@/services/stream.js';
 import { fetchMeta } from '@/misc/fetch-meta.js';
@@ -290,25 +291,36 @@ async function upload(key: string, stream: fs.ReadStream | Buffer, _type: string
 	if (result) logger.debug(`Uploaded: ${result.Bucket}/${result.Key} => ${result.Location}`);
 }
 
-async function deleteOldFile(user: IRemoteUser): Promise<void> {
-	const q = DriveFiles.createQueryBuilder('file')
-		.where('file.userId = :userId', { userId: user.id })
-		.andWhere('NOT file.isLink');
+async function expireOldFiles(user: IRemoteUser, driveCapacity: number): Promise<void> {
+	// Delete as many files as necessary so the total usage is below driveCapacity,
+	// oldest files first, and exclude avatar and banner.
+	//
+	// Using a window function, i.e. `OVER (ORDER BY "createdAt" DESC)` means that
+	// the `SUM` will be a running total.
+	const exceededFileIds = await db.query('SELECT "id" FROM ('
+		+ 'SELECT "id", SUM("size") OVER (ORDER BY "createdAt" DESC) AS "total" FROM "drive_file" WHERE "userId" = $1 AND NOT "isLink"'
+		+ (user.avatarId ? ' AND "id" != $2' : '')
+		+ (user.bannerId ? ' AND "id" != $3' : '')
+		+ ') AS "totals" WHERE "total" > $4',
+		[
+			user.id,
+			user.avatarId ?? '',
+			user.bannerId ?? '',
+			driveCapacity,
+		]
+	);
 
-	if (user.avatarId) {
-		q.andWhere('file.id != :avatarId', { avatarId: user.avatarId });
+	if (exceededFileIds.length === 0) {
+		// no files to expire, avatar and banner if present are already the only files
+		throw new Error('remote user drive quota met by avatar and banner');
 	}
 
-	if (user.bannerId) {
-		q.andWhere('file.id != :bannerId', { bannerId: user.bannerId });
-	}
+	const files = await DriveFiles.findBy({
+		id: In(exceededFileIds.map(x => x.id)),
+	});
 
-	q.orderBy('file.id', 'ASC');
-
-	const oldFile = await q.getOne();
-
-	if (oldFile) {
-		deleteFile(oldFile, true);
+	for (const file of files) {
+		deleteFile(file, true);
 	}
 }
 
@@ -373,19 +385,20 @@ export async function addFile({
 	//#region Check drive usage
 	if (user && !isLink) {
 		const usage = await DriveFiles.calcDriveUsageOf(user.id);
+		const isLocalUser = Users.isLocalUser(user);
 
 		const instance = await fetchMeta();
-		const driveCapacity = 1024 * 1024 * (Users.isLocalUser(user) ? instance.localDriveCapacityMb : instance.remoteDriveCapacityMb);
+		const driveCapacity = 1024 * 1024 * (isLocalUser ? instance.localDriveCapacityMb : instance.remoteDriveCapacityMb);
 
 		logger.debug(`drive usage is ${usage} (max: ${driveCapacity})`);
 
 		// If usage limit exceeded
 		if (usage + info.size > driveCapacity) {
-			if (Users.isLocalUser(user)) {
+			if (isLocalUser) {
 				throw new Error('no-free-space');
 			} else {
-				// delete oldest file (excluding banner and avatar)
-				deleteOldFile(await Users.findOneByOrFail({ id: user.id }) as IRemoteUser);
+				// delete older files to make space for new file
+				expireOldFiles(await Users.findOneByOrFail({ id: user.id }) as IRemoteUser, driveCapacity - info.size);
 			}
 		}
 	}

packages/backend/src/services/drive/delete-file.ts

@@ -60,14 +60,14 @@ export async function deleteFileSync(file: DriveFile, isExpired = false): Promis
 		await Promise.all(promises);
 	}
 
-	postProcess(file, isExpired);
+	await postProcess(file, isExpired);
 }
 
 async function postProcess(file: DriveFile, isExpired = false): Promise<void> {
 	// Turn into a direct link after expiring a remote file.
 	if (isExpired && file.userHost != null && file.uri != null) {
 		const id = uuid();
-		DriveFiles.update(file.id, {
+		await DriveFiles.update(file.id, {
 			isLink: true,
 			url: file.uri,
 			thumbnailUrl: null,
@@ -78,14 +78,14 @@ async function postProcess(file: DriveFile, isExpired = false): Promise<void> {
 			webpublicAccessKey: 'webpublic-' + id,
 		});
 	} else {
-		DriveFiles.delete(file.id);
+		await DriveFiles.delete(file.id);
 	}
 
 	// update statistics
-	driveChart.update(file, false);
-	perUserDriveChart.update(file, false);
+	await driveChart.update(file, false);
+	await perUserDriveChart.update(file, false);
 	if (file.userHost != null) {
-		instanceChart.updateDrive(file, false);
+		await instanceChart.updateDrive(file, false);
 	}
 }
 

packages/backend/src/services/logger.ts

@@ -3,7 +3,7 @@ import chalk from 'chalk';
 import convertColor from 'color-convert';
 import { format as dateFormat } from 'date-fns';
 import config from '@/config/index.js';
-import { envOption } from '@/env.js';
+import { envOption, LOG_LEVELS } from '@/env.js';
 import type { KEYWORD } from 'color-convert/conversions.js';
 
 type Domain = {
@@ -11,14 +11,7 @@ type Domain = {
 	color?: KEYWORD;
 };
 
-export const LEVELS = {
-	error: 0,
-	warning: 1,
-	success: 2,
-	info: 3,
-	debug: 4,
-};
-export type Level = LEVELS[keyof LEVELS];
+export type Level = LOG_LEVELS[keyof LOG_LEVELS];
 
 /**
  * Class that facilitates recording log messages to the console.
@@ -38,7 +31,7 @@ export default class Logger {
 	 * @param color Log message color
 	 * @param store Whether to store messages
 	 */
-	constructor(domain: string, color?: KEYWORD, store = true, minLevel: Level = LEVELS.info) {
+	constructor(domain: string, color?: KEYWORD, store = true, minLevel: Level = LOG_LEVELS.info) {
 		this.domain = {
 			name: domain,
 			color,
@@ -54,7 +47,7 @@ export default class Logger {
 	 * @param store Whether to store messages
 	 * @returns A Logger instance whose parent logger is this instance.
 	 */
-	public createSubLogger(domain: string, color?: KEYWORD, store = true, minLevel: Level = LEVELS.info): Logger {
+	public createSubLogger(domain: string, color?: KEYWORD, store = true, minLevel: Level = LOG_LEVELS.info): Logger {
 		const logger = new Logger(domain, color, store, minLevel);
 		logger.parentLogger = this;
 		return logger;
@@ -69,7 +62,6 @@ export default class Logger {
 	 * @param subDomains Names of sub-loggers to be added.
 	 */
 	private log(level: Level, message: string, important = false, subDomains: Domain[] = [], _store = true): void {
-		if (envOption.quiet) return;
 		const store = _store && this.store;
 
 		// Check against the configured log level.
@@ -89,7 +81,7 @@ export default class Logger {
 		let levelDisplay;
 		let messageDisplay;
 		switch (level) {
-			case LEVELS.error:
+			case LOG_LEVELS.error:
 				if (important) {
 					levelDisplay = chalk.bgRed.white('ERR ');
 				} else {
@@ -97,11 +89,11 @@ export default class Logger {
 				}
 				messageDisplay = chalk.red(message);
 				break;
-			case LEVELS.warning:
+			case LOG_LEVELS.warning:
 				levelDisplay = chalk.yellow('WARN');
 				messageDisplay = chalk.yellow(message);
 				break;
-			case LEVELS.success:
+			case LOG_LEVELS.success:
 				if (important) {
 					levelDisplay = chalk.bgGreen.white('DONE');
 				} else {
@@ -109,11 +101,11 @@ export default class Logger {
 				}
 				messageDisplay = chalk.green(message);
 				break;
-			case LEVELS.info:
+			case LOG_LEVELS.info:
 				levelDisplay = chalk.blue('INFO');
 				messageDisplay = message;
 				break;
-			case LEVELS.debug: default:
+			case LOG_LEVELS.debug: default:
 				levelDisplay = chalk.gray('VERB');
 				messageDisplay = chalk.gray(message);
 				break;
@@ -133,11 +125,11 @@ export default class Logger {
 	 */
 	public error(err: string | Error, important = false): void {
 		if (err instanceof Error) {
-			this.log(LEVELS.error, err.toString(), important);
+			this.log(LOG_LEVELS.error, err.toString(), important);
 		} else if (typeof err === 'object') {
-			this.log(LEVELS.error, `${(err as any).message || (err as any).name || err}`, important);
+			this.log(LOG_LEVELS.error, `${(err as any).message || (err as any).name || err}`, important);
 		} else {
-			this.log(LEVELS.error, `${err}`, important);
+			this.log(LOG_LEVELS.error, `${err}`, important);
 		}
 	}
 
@@ -148,7 +140,7 @@ export default class Logger {
 	 * @param important Whether this warning is important
 	 */
 	public warn(message: string, important = false): void {
-		this.log(LEVELS.warning, message, important);
+		this.log(LOG_LEVELS.warning, message, important);
 	}
 
 	/**
@@ -158,7 +150,7 @@ export default class Logger {
 	 * @param important Whether this success message is important
 	 */
 	public succ(message: string, important = false): void {
-		this.log(LEVELS.success, message, important);
+		this.log(LOG_LEVELS.success, message, important);
 	}
 
 	/**
@@ -168,7 +160,7 @@ export default class Logger {
 	 * @param important Whether this debug message is important
 	 */
 	public debug(message: string, important = false): void {
-		this.log(LEVELS.debug, message, important);
+		this.log(LOG_LEVELS.debug, message, important);
 	}
 
 	/**
@@ -178,6 +170,6 @@ export default class Logger {
 	 * @param important Whether this info message is important
 	 */
 	public info(message: string, important = false): void {
-		this.log(LEVELS.info, message, important);
+		this.log(LOG_LEVELS.info, message, important);
 	}
 }

packages/backend/src/services/suspend-user.ts

@@ -6,6 +6,9 @@ import { User } from '@/models/entities/user.js';
 import { Users } from '@/models/index.js';
 import { publishInternalEvent } from '@/services/stream.js';
 
+/**
+ * Sends an internal event and for local users queues the delete activites.
+ */
 export async function doPostSuspend(user: { id: User['id']; host: User['host'] }): Promise<void> {
 	publishInternalEvent('userChangeSuspendedState', { id: user.id, isSuspended: true });
 
@@ -15,6 +18,6 @@ export async function doPostSuspend(user: { id: User['id']; host: User['host'] }
 		// deliver to all of known network
 		const dm = new DeliverManager(user, content);
 		dm.addEveryone();
-		await dm.execute();
+		await dm.execute(user.id);
 	}
 }

packages/backend/src/services/user-cache.ts

@@ -6,15 +6,15 @@ import { subscriber } from '@/db/redis.js';
 
 export const userByIdCache = new Cache<User>(
 	Infinity,
-	async (id) => await Users.findOneBy({ id, isDeleted: false }) ?? undefined,
+	async (id) => await Users.findOneBy({ id, isDeleted: IsNull() }) ?? undefined,
 );
 export const localUserByNativeTokenCache = new Cache<ILocalUser>(
 	Infinity,
-	async (token) => await Users.findOneBy({ token, host: IsNull(), isDeleted: false }) as ILocalUser | null ?? undefined,
+	async (token) => await Users.findOneBy({ token, host: IsNull(), isDeleted: IsNull() }) as ILocalUser | null ?? undefined,
 );
 export const uriPersonCache = new Cache<User>(
 	Infinity,
-	async (uri) => await Users.findOneBy({ uri, isDeleted: false }) ?? undefined,
+	async (uri) => await Users.findOneBy({ uri, isDeleted: IsNull() }) ?? undefined,
 );
 
 subscriber.on('message', async (_, data) => {

packages/client/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "client",
-	"version": "13.0.0-preview4",
+	"version": "13.0.0-preview5",
 	"private": true,
 	"scripts": {
 		"watch": "vite build --watch --mode development",

packages/client/src/menu.ts

@@ -120,12 +120,6 @@ export const menuDef = reactive({
 		indicated: computed(() => $i != null && $i.hasUnreadSpecifiedNotes),
 		to: '/my/notifications#directNotes',
 	},
-	favorites: {
-		title: 'favorites',
-		icon: 'fas fa-star',
-		show: computed(() => $i != null),
-		to: '/my/favorites',
-	},
 	pages: {
 		title: 'pages',
 		icon: 'fas fa-file-alt',

packages/client/src/pages/favorites.vue

@@ -1,50 +0,0 @@
-<template>
-<MkStickyContainer>
-	<template #header><MkPageHeader/></template>
-	<MkSpacer :content-max="800">
-		<MkPagination ref="pagingComponent" :pagination="pagination">
-			<template #empty>
-				<div class="_fullinfo">
-					<img :src="instance.images.info" class="_ghost"/>
-					<div>{{ i18n.ts.noNotes }}</div>
-				</div>
-			</template>
-
-			<template #default="{ items }">
-				<XList v-slot="{ item }" :items="items" :direction="'down'" :no-gap="false">
-					<XNote :key="item.id" :note="item.note" :class="$style.note"/>
-				</XList>
-			</template>
-		</MkPagination>
-	</MkSpacer>
-</MkStickyContainer>
-</template>
-
-<script lang="ts" setup>
-import { ref } from 'vue';
-import MkPagination from '@/components/ui/pagination.vue';
-import XNote from '@/components/note.vue';
-import XList from '@/components/date-separated-list.vue';
-import { i18n } from '@/i18n';
-import { definePageMetadata } from '@/scripts/page-metadata';
-import { instance } from '@/instance';
-
-const pagination = {
-	endpoint: 'i/favorites' as const,
-	limit: 10,
-};
-
-const pagingComponent = ref<InstanceType<typeof MkPagination>>();
-
-definePageMetadata({
-	title: i18n.ts.favorites,
-	icon: 'fas fa-star',
-});
-</script>
-
-<style lang="scss" module>
-.note {
-	background: var(--panel);
-	border-radius: var(--radius);
-}
-</style>

packages/client/src/router.ts

@@ -148,10 +148,6 @@ export const routes = [{
 	component: page(() => import('./pages/notifications.vue')),
 	hash: 'initialTab',
 	loginRequired: true,
-}, {
-	path: '/my/favorites',
-	component: page(() => import('./pages/favorites.vue')),
-	loginRequired: true,
 }, {
 	name: 'messaging',
 	path: '/my/messaging',

packages/client/src/scripts/get-note-menu.ts

@@ -48,12 +48,6 @@ export function getNoteMenu(props: {
 		});
 	}
 
-	function toggleFavorite(favorite: boolean): void {
-		os.apiWithDialog(favorite ? 'notes/favorites/create' : 'notes/favorites/delete', {
-			noteId: appearNote.id,
-		});
-	}
-
 	function toggleWatch(watch: boolean): void {
 		os.apiWithDialog(watch ? 'notes/watching/create' : 'notes/watching/delete', {
 			noteId: appearNote.id,
@@ -244,15 +238,6 @@ export function getNoteMenu(props: {
 				action: translate,
 			} : undefined,
 			null,
-			statePromise.then(state => state.isFavorited ? {
-				icon: 'fas fa-star',
-				text: i18n.ts.unfavorite,
-				action: () => toggleFavorite(false),
-			} : {
-				icon: 'fas fa-star',
-				text: i18n.ts.favorite,
-				action: () => toggleFavorite(true),
-			}),
 			{
 				icon: 'fas fa-paperclip',
 				text: i18n.ts.clip,

packages/client/src/store.ts

@@ -59,7 +59,6 @@ export const defaultStore = markRaw(new Storage('base', {
 		where: 'deviceAccount',
 		default: [
 			'notifications',
-			'favorites',
 			'drive',
 			'followRequests',
 			'-',

packages/foundkey-js/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "foundkey-js",
-	"version": "13.0.0-preview4",
+	"version": "13.0.0-preview5",
 	"description": "Fork of misskey-js for Foundkey",
 	"type": "module",
 	"main": "./built/index.js",

packages/foundkey-js/src/api.types.ts

@@ -1,7 +1,7 @@
 import {
 	Announcement, Antenna, App, AuthSession, Blocking, Channel, Clip, DateString, InstanceMetadata, DriveFile, DriveFolder, Following, FollowingFolloweePopulated, FollowingFollowerPopulated, FollowRequest, Instance,
 	MeDetailed,
-	Note, NoteFavorite, OriginType, Page, ServerInfo, Stats, User, UserDetailed, UserGroup, UserList, UserSorting, Notification, NoteReaction, Signin, MessagingMessage,
+	Note, OriginType, Page, ServerInfo, Stats, User, UserDetailed, UserGroup, UserList, UserSorting, Notification, NoteReaction, Signin, MessagingMessage,
 } from './entities.js';
 
 type TODO = Record<string, any> | null;
@@ -304,7 +304,6 @@ export type Endpoints = {
 	'i/export-mute': { req: TODO; res: TODO; };
 	'i/export-notes': { req: TODO; res: TODO; };
 	'i/export-user-lists': { req: TODO; res: TODO; };
-	'i/favorites': { req: { limit?: number; sinceId?: NoteFavorite['id']; untilId?: NoteFavorite['id']; }; res: NoteFavorite[]; };
 	'i/get-word-muted-notes-count': { req: TODO; res: TODO; };
 	'i/import-blocking': { req: TODO; res: TODO; };
 	'i/import-following': { req: TODO; res: TODO; };
@@ -411,8 +410,6 @@ export type Endpoints = {
 		};
 	}; res: { createdNote: Note }; };
 	'notes/delete': { req: { noteId: Note['id']; }; res: null; };
-	'notes/favorites/create': { req: { noteId: Note['id']; }; res: null; };
-	'notes/favorites/delete': { req: { noteId: Note['id']; }; res: null; };
 	'notes/featured': { req: TODO; res: Note[]; };
 	'notes/global-timeline': { req: { limit?: number; sinceId?: Note['id']; untilId?: Note['id']; sinceDate?: number; untilDate?: number; }; res: Note[]; };
 	'notes/hybrid-timeline': { req: { limit?: number; sinceId?: Note['id']; untilId?: Note['id']; sinceDate?: number; untilDate?: number; }; res: Note[]; };

packages/foundkey-js/src/consts.ts

@@ -13,8 +13,6 @@ export const permissions = [
 	'write:blocks',
 	'read:drive',
 	'write:drive',
-	'read:favorites',
-	'write:favorites',
 	'read:following',
 	'write:following',
 	'read:messaging',

packages/foundkey-js/src/entities.ts

@@ -385,13 +385,6 @@ export type AuthSession = {
 
 export type Clip = TODO;
 
-export type NoteFavorite = {
-	id: ID;
-	createdAt: DateString;
-	noteId: Note['id'];
-	note: Note;
-};
-
 export type FollowRequest = {
 	id: ID;
 	follower: User;

packages/sw/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "sw",
-	"version": "13.0.0-preview4",
+	"version": "13.0.0-preview5",
 	"private": true,
 	"scripts": {
 		"watch": "node build.js watch",