Bump version to 3.2.1

* Add support for latest HTTP Signatures spec draft

https://www.ietf.org/id/draft-ietf-httpbis-message-signatures-00.html

- add support for the “hs2019” signature algorithm (assumed to be equivalent
  to RSA-SHA256, since we do not have a mechanism to specify the algorithm
  within the key metadata yet)
- add support for (created) and (expires) pseudo-headers and related
  signature parameters, when using the hs2019 signature algorithm
- adjust default “headers” parameter while being backwards-compatible with
  previous implementation
- change the acceptable time window logic from 12 hours surrounding the “date”
  header to accepting signatures created up to 1 hour in the future and
  expiring up to 1 hour in the past (but only allowing expiration dates up to
  12 hours after the creation date)
  This doesn't conform with the current draft, as it doesn't permit accounting
  for clock skew.
  This, however, should be addressed in a next version of the draft:
  https://github.com/httpwg/http-extensions/pull/1235

* Add additional signature requirements

* Rewrite signature params parsing using Parslet

* Make apparent which signature algorithm Mastodon on verification failure

Mastodon uses RSASSA-PKCS1-v1_5, which is not recommended for new applications,
and new implementers may thus unknowingly use RSASSA-PSS.

* Add workaround for PeerTube's invalid signature header

The previous parser allowed incorrect Signature headers, such as
those produced by old versions of the `http-signature` node.js package,
and seemingly used by PeerTube.

This commit adds a workaround for that.

* Fix `signature_key_id` raising an exception

Previously, parsing failures would result in `signature_key_id` being nil,
but the parser changes made that result in an exception.

This commit changes the `signature_key_id` method to return `nil` in case
of parsing failures.

* Move extra HTTP signature helper methods to private methods

* Relax (request-target) requirement to (request-target) || digest

This lets requests from Plume work without lowering security significantly.

.buildpacks

@@ -1,4 +1,3 @@
 https://github.com/heroku/heroku-buildpack-apt
 https://github.com/Scalingo/ffmpeg-buildpack
-https://github.com/Scalingo/nodejs-buildpack
 https://github.com/Scalingo/ruby-buildpack

.circleci/config.yml

@@ -3,13 +3,15 @@ version: 2
 aliases:
   - &defaults
     docker:
-      - image: circleci/ruby:2.5.1-stretch-node
+      - image: circleci/ruby:2.7-buster-node
         environment: &ruby_environment
+          BUNDLE_JOBS: 3
+          BUNDLE_RETRY: 3
           BUNDLE_APP_CONFIG: ./.bundle/
+          BUNDLE_PATH: ./vendor/bundle/
           DB_HOST: localhost
           DB_USER: root
           RAILS_ENV: test
-          PARALLEL_TEST_PROCESSORS: 4
           ALLOW_NOPAM: true
           CONTINUOUS_INTEGRATION: true
           DISABLE_SIMPLECOV: true
@@ -31,25 +33,25 @@ aliases:
   - &restore_ruby_dependencies
     restore_cache:
       keys:
-        - v2-ruby-dependencies-{{ checksum "/tmp/.ruby-version" }}-{{ checksum "Gemfile.lock" }}
-        - v2-ruby-dependencies-{{ checksum "/tmp/.ruby-version" }}-
-        - v2-ruby-dependencies-
+        - v3-ruby-dependencies-{{ checksum "/tmp/.ruby-version" }}-{{ checksum "Gemfile.lock" }}
+        - v3-ruby-dependencies-{{ checksum "/tmp/.ruby-version" }}-
+        - v3-ruby-dependencies-
 
   - &install_steps
     steps:
       - checkout
       - *attach_workspace
-
       - restore_cache:
           keys:
-            - v1-node-dependencies-{{ checksum "yarn.lock" }}
-            - v1-node-dependencies-
-      - run: yarn install --frozen-lockfile
+            - v2-node-dependencies-{{ checksum "yarn.lock" }}
+            - v2-node-dependencies-
+      - run:
+          name: Install yarn dependencies
+          command: yarn install --frozen-lockfile
       - save_cache:
-          key: v1-node-dependencies-{{ checksum "yarn.lock" }}
+          key: v2-node-dependencies-{{ checksum "yarn.lock" }}
           paths:
             - ./node_modules/
-
       - *persist_to_workspace
 
   - &install_system_dependencies
@@ -62,14 +64,25 @@ aliases:
   - &install_ruby_dependencies
       steps:
         - *attach_workspace
-
         - *install_system_dependencies
-
-        - run: ruby -e 'puts RUBY_VERSION' | tee /tmp/.ruby-version
+        - run:
+            name: Set Ruby version
+            command: ruby -e 'puts RUBY_VERSION' | tee /tmp/.ruby-version
         - *restore_ruby_dependencies
-        - run: bundle install --clean --jobs 16 --path ./vendor/bundle/ --retry 3 --with pam_authentication --without development production && bundle clean
+        - run:
+            name: Set bundler settings
+            command: |
+              bundle config --local clean 'true'
+              bundle config --local deployment 'true'
+              bundle config --local with 'pam_authentication'
+              bundle config --local without 'development production'
+              bundle config --local frozen 'true'
+              bundle config --local path $BUNDLE_PATH
+        - run:
+            name: Install bundler dependencies
+            command: bundle check || (bundle install && bundle clean)
         - save_cache:
-            key: v2-ruby-dependencies-{{ checksum "/tmp/.ruby-version" }}-{{ checksum "Gemfile.lock" }}
+            key: v3-ruby-dependencies-{{ checksum "/tmp/.ruby-version" }}-{{ checksum "Gemfile.lock" }}
             paths:
               - ./.bundle/
               - ./vendor/bundle/
@@ -80,39 +93,39 @@ aliases:
                 - ./mastodon/vendor/bundle/
 
   - &test_steps
+      parallelism: 4
       steps:
         - *attach_workspace
-
         - *install_system_dependencies
-        - run: sudo apt-get install -y ffmpeg
-
         - run:
-            name: Prepare Tests
-            command: ./bin/rails parallel:create parallel:load_schema parallel:prepare
+            name: Install FFMPEG
+            command: sudo apt-get install -y ffmpeg
         - run:
-            name: Run Tests
-            command: ./bin/retry bundle exec parallel_test ./spec/ --group-by filesize --type rspec
-
+            name: Load database schema
+            command: ./bin/rails db:create db:schema:load db:seed
+        - run:
+            name: Run rspec in parallel
+            command: |
+              bundle exec rspec --profile 10 \
+                                --format RspecJunitFormatter \
+                                --out test_results/rspec.xml \
+                                --format progress \
+                                $(circleci tests glob "spec/**/*_spec.rb" | circleci tests split --split-by=timings)
+        - store_test_results:
+            path: test_results
 jobs:
   install:
     <<: *defaults
     <<: *install_steps
 
-  install-ruby2.5:
+  install-ruby2.7:
     <<: *defaults
     <<: *install_ruby_dependencies
 
-  install-ruby2.4:
+  install-ruby2.6:
     <<: *defaults
     docker:
-      - image: circleci/ruby:2.4.4-stretch-node
-        environment: *ruby_environment
-    <<: *install_ruby_dependencies
-
-  install-ruby2.3:
-    <<: *defaults
-    docker:
-      - image: circleci/ruby:2.3.7-stretch-node
+      - image: circleci/ruby:2.6-buster-node
         environment: *ruby_environment
     <<: *install_ruby_dependencies
 
@@ -121,97 +134,116 @@ jobs:
     steps:
       - *attach_workspace
       - *install_system_dependencies
-      - run: ./bin/rails assets:precompile
+      - run:
+          name: Precompile assets
+          command: ./bin/rails assets:precompile
       - persist_to_workspace:
           root: ~/projects/
           paths:
               - ./mastodon/public/assets
               - ./mastodon/public/packs-test/
 
-  test-ruby2.5:
+  test-migrations:
     <<: *defaults
     docker:
-      - image: circleci/ruby:2.5.1-stretch-node
+      - image: circleci/ruby:2.7-buster-node
         environment: *ruby_environment
-      - image: circleci/postgres:10.3-alpine
+      - image: circleci/postgres:12.2
         environment:
           POSTGRES_USER: root
-      - image: circleci/redis:4.0.9-alpine
+          POSTGRES_HOST_AUTH_METHOD: trust
+      - image: circleci/redis:5-alpine
+    steps:
+      - *attach_workspace
+      - *install_system_dependencies
+      - run:
+          name: Create database
+          command: ./bin/rails db:create
+      - run:
+          name: Run migrations
+          command: ./bin/rails db:migrate
+
+  test-ruby2.7:
+    <<: *defaults
+    docker:
+      - image: circleci/ruby:2.7-buster-node
+        environment: *ruby_environment
+      - image: circleci/postgres:12.2
+        environment:
+          POSTGRES_USER: root
+          POSTGRES_HOST_AUTH_METHOD: trust
+      - image: circleci/redis:5-alpine
     <<: *test_steps
 
-  test-ruby2.4:
+  test-ruby2.6:
     <<: *defaults
     docker:
-      - image: circleci/ruby:2.4.4-stretch-node
+      - image: circleci/ruby:2.6-buster-node
         environment: *ruby_environment
-      - image: circleci/postgres:10.3-alpine
+      - image: circleci/postgres:12.2
         environment:
           POSTGRES_USER: root
-      - image: circleci/redis:4.0.9-alpine
-    <<: *test_steps
-
-  test-ruby2.3:
-    <<: *defaults
-    docker:
-      - image: circleci/ruby:2.3.7-stretch-node
-        environment: *ruby_environment
-      - image: circleci/postgres:10.3-alpine
-        environment:
-          POSTGRES_USER: root
-      - image: circleci/redis:4.0.9-alpine
+          POSTGRES_HOST_AUTH_METHOD: trust
+      - image: circleci/redis:5-alpine
     <<: *test_steps
 
   test-webui:
     <<: *defaults
     docker:
-      - image: circleci/node:8.11.1-stretch
+      - image: circleci/node:12-buster
     steps:
       - *attach_workspace
-      - run: ./bin/retry yarn test:jest
+      - run:
+          name: Run jest
+          command: yarn test:jest
 
   check-i18n:
     <<: *defaults
     steps:
       - *attach_workspace
-      - run: bundle exec i18n-tasks check-normalized
-      - run: bundle exec i18n-tasks unused
-      - run: bundle exec i18n-tasks missing -t plural
-      - run: bundle exec i18n-tasks check-consistent-interpolations
+      - *install_system_dependencies
+      - run:
+          name: Check locale file normalization
+          command: bundle exec i18n-tasks check-normalized
+      - run:
+          name: Check for unused strings
+          command: bundle exec i18n-tasks unused -l en
+      - run:
+          name: Check for wrong string interpolations
+          command: bundle exec i18n-tasks check-consistent-interpolations
+      - run:
+          name: Check that all required locale files exist
+          command: bundle exec rake repo:check_locales_files
 
 workflows:
   version: 2
   build-and-test:
     jobs:
       - install
-      - install-ruby2.5:
+      - install-ruby2.7:
           requires:
             - install
-      - install-ruby2.4:
+      - install-ruby2.6:
           requires:
             - install
-            - install-ruby2.5
-      - install-ruby2.3:
-          requires:
-            - install
-            - install-ruby2.5
+            - install-ruby2.7
       - build:
           requires:
-            - install-ruby2.5
-      - test-ruby2.5:
+            - install-ruby2.7
+      - test-migrations:
           requires:
-            - install-ruby2.5
+            - install-ruby2.7
+      - test-ruby2.7:
+          requires:
+            - install-ruby2.7
             - build
-      - test-ruby2.4:
+      - test-ruby2.6:
           requires:
-            - install-ruby2.4
-            - build
-      - test-ruby2.3:
-          requires:
-            - install-ruby2.3
+            - install-ruby2.6
             - build
       - test-webui:
           requires:
             - install
       - check-i18n:
           requires:
-            - install-ruby2.5
+            - install-ruby2.7

.codeclimate.yml

@@ -27,11 +27,11 @@ plugins:
     enabled: true
   eslint:
     enabled: true
-    channel: eslint-4
+    channel: eslint-6
   rubocop:
     enabled: true
-    channel: rubocop-0-54
-  scss-lint:
+    channel: rubocop-0-82
+  sass-lint:
     enabled: true
 exclude_patterns:
 - spec/

.env.nanobox

@@ -11,24 +11,14 @@ DB_NAME=gonano
 DB_PASS=$DATA_DB_PASS
 DB_PORT=5432
 
-DATABASE_URL=postgresql://$DATA_DB_USER:$DATA_DB_PASS@$DATA_DB_HOST/gonano
+# DATABASE_URL=postgresql://$DATA_DB_USER:$DATA_DB_PASS@$DATA_DB_HOST/gonano
 
 # Optional ElasticSearch configuration
 ES_ENABLED=true
 ES_HOST=$DATA_ELASTIC_HOST
 ES_PORT=9200
 
-# Optimizations
-LD_PRELOAD=/data/lib/libjemalloc.so
-
-# ImageMagick optimizations
-MAGICK_TEMPORARY_PATH=/app/tmp
-MAGICK_MEMORY_LIMIT=128MiB
-MAGICK_MAP_LIMIT=64MiB
-MAGICK_TIME_LIMIT=15
-MAGICK_AREA_LIMIT=16MP
-MAGICK_WIDTH_LIMIT=8KP
-MAGICK_HEIGHT_LIMIT=8KP
+BIND=0.0.0.0
 
 # Federation
 # Note: Changing LOCAL_DOMAIN at a later time will cause unwanted side effects, including breaking all existing federation.
@@ -84,6 +74,7 @@ SMTP_PORT=587
 SMTP_LOGIN=$SMTP_LOGIN
 SMTP_PASSWORD=$SMTP_PASSWORD
 SMTP_FROM_ADDRESS=notifications@${APP_NAME}.nanoapp.io
+#SMTP_REPLY_TO=
 #SMTP_DOMAIN= # defaults to LOCAL_DOMAIN
 #SMTP_DELIVERY_METHOD=smtp # delivery method can also be sendmail
 #SMTP_AUTH_METHOD=plain
@@ -97,9 +88,17 @@ SMTP_FROM_ADDRESS=notifications@${APP_NAME}.nanoapp.io
 # PAPERCLIP_ROOT_URL=/system
 
 # Optional asset host for multi-server setups
+# The asset host must allow cross origin request from WEB_DOMAIN or LOCAL_DOMAIN
+# if WEB_DOMAIN is not set. For example, the server may have the
+# following header field:
+# Access-Control-Allow-Origin: https://example.com/
 # CDN_HOST=https://assets.example.com
 
 # S3 (optional)
+# The attachment host must allow cross origin request from WEB_DOMAIN or
+# LOCAL_DOMAIN if WEB_DOMAIN is not set. For example, the server may have the
+# following header field:
+# Access-Control-Allow-Origin: https://192.168.1.123:9000/
 # S3_ENABLED=true
 # S3_BUCKET=
 # AWS_ACCESS_KEY_ID=
@@ -109,6 +108,8 @@ SMTP_FROM_ADDRESS=notifications@${APP_NAME}.nanoapp.io
 # S3_HOSTNAME=192.168.1.123:9000
 
 # S3 (Minio Config (optional) Please check Minio instance for details)
+# The attachment host must allow cross origin request - see the description
+# above.
 # S3_ENABLED=true
 # S3_BUCKET=
 # AWS_ACCESS_KEY_ID=
@@ -119,12 +120,30 @@ SMTP_FROM_ADDRESS=notifications@${APP_NAME}.nanoapp.io
 # S3_ENDPOINT=
 # S3_SIGNATURE_VERSION=
 
+# Google Cloud Storage (optional)
+# Use S3 compatible API. Since GCS does not support Multipart Upload,
+# increase the value of S3_MULTIPART_THRESHOLD to disable Multipart Upload.
+# The attachment host must allow cross origin request - see the description
+# above.
+# S3_ENABLED=true
+# AWS_ACCESS_KEY_ID=
+# AWS_SECRET_ACCESS_KEY=
+# S3_REGION=
+# S3_PROTOCOL=https
+# S3_HOSTNAME=storage.googleapis.com
+# S3_ENDPOINT=https://storage.googleapis.com
+# S3_MULTIPART_THRESHOLD=52428801 # 50.megabytes
+
 # Swift (optional)
+# The attachment host must allow cross origin request - see the description
+# above.
 # SWIFT_ENABLED=true
 # SWIFT_USERNAME=
 # For Keystone V3, the value for SWIFT_TENANT should be the project name
 # SWIFT_TENANT=
 # SWIFT_PASSWORD=
+# Some OpenStack V3 providers require PROJECT_ID (optional)
+# SWIFT_PROJECT_ID=
 # Keystone V2 and V3 URLs are supported. Use a V3 URL if possible to avoid
 # issues with token rate-limiting during high load.
 # SWIFT_AUTH_URL=
@@ -164,6 +183,11 @@ SMTP_FROM_ADDRESS=notifications@${APP_NAME}.nanoapp.io
 # LDAP_BIND_DN=
 # LDAP_PASSWORD=
 # LDAP_UID=cn
+# LDAP_MAIL=mail
+# LDAP_SEARCH_FILTER=(|(%{uid}=%{email})(%{mail}=%{email}))
+# LDAP_UID_CONVERSION_ENABLED=true
+# LDAP_UID_CONVERSION_SEARCH=., -
+# LDAP_UID_CONVERSION_REPLACE=_
 
 # PAM authentication (optional)
 # PAM authentication uses for the email generation the "email" pam variable
@@ -171,8 +195,8 @@ SMTP_FROM_ADDRESS=notifications@${APP_NAME}.nanoapp.io
 # The pam environment variable "email" is provided by:
 # https://github.com/devkral/pam_email_extractor
 # PAM_ENABLED=true
-# Fallback Suffix for email address generation (nil by default)
-# PAM_DEFAULT_SUFFIX=pam
+# Fallback email domain for email address generation (LOCAL_DOMAIN by default)
+# PAM_EMAIL_DOMAIN=example.com
 # Name of the pam service (pam "auth" section is evaluated)
 # PAM_DEFAULT_SERVICE=rpam
 # Name of the pam service used for checking if an user can register (pam "account" section is evaluated) (nil (disabled) by default)
@@ -207,8 +231,8 @@ SMTP_FROM_ADDRESS=notifications@${APP_NAME}.nanoapp.io
 
 # Optional SAML authentication (cf. omniauth-saml)
 # SAML_ENABLED=true
-# SAML_ACS_URL=
-# SAML_ISSUER=http://localhost:3000/auth/auth/saml/callback
+# SAML_ACS_URL=http://localhost:3000/auth/auth/saml/callback
+# SAML_ISSUER=https://example.com
 # SAML_IDP_SSO_TARGET_URL=https://idp.testshib.org/idp/profile/SAML2/Redirect/SSO
 # SAML_IDP_CERT=
 # SAML_IDP_CERT_FINGERPRINT=
@@ -220,7 +244,14 @@ SMTP_FROM_ADDRESS=notifications@${APP_NAME}.nanoapp.io
 # SAML_SECURITY_ASSUME_EMAIL_IS_VERIFIED=true
 # SAML_ATTRIBUTES_STATEMENTS_UID="urn:oid:0.9.2342.19200300.100.1.1"
 # SAML_ATTRIBUTES_STATEMENTS_EMAIL="urn:oid:1.3.6.1.4.1.5923.1.1.1.6"
-# SAML_ATTRIBUTES_STATEMENTS_FULL_NAME="urn:oid:2.5.4.42"
+# SAML_ATTRIBUTES_STATEMENTS_FULL_NAME="urn:oid:2.16.840.1.113730.3.1.241"
+# SAML_ATTRIBUTES_STATEMENTS_FIRST_NAME="urn:oid:2.5.4.42"
+# SAML_ATTRIBUTES_STATEMENTS_LAST_NAME="urn:oid:2.5.4.4"
 # SAML_UID_ATTRIBUTE="urn:oid:0.9.2342.19200300.100.1.1"
 # SAML_ATTRIBUTES_STATEMENTS_VERIFIED=
 # SAML_ATTRIBUTES_STATEMENTS_VERIFIED_EMAIL=
+
+# Use HTTP proxy for outgoing request (optional)
+# http_proxy=http://gateway.local:8118
+# Access control for hidden service.
+# ALLOW_ACCESS_TO_HIDDEN_SERVICE=true

.env.production.sample

@@ -1,232 +1,60 @@
-# Service dependencies
-# You may set REDIS_URL instead for more advanced options
-# You may also set REDIS_NAMESPACE to share Redis between multiple Mastodon servers
-REDIS_HOST=redis
-REDIS_PORT=6379
-# You may set DATABASE_URL instead for more advanced options
-DB_HOST=db
-DB_USER=postgres
-DB_NAME=postgres
-DB_PASS=
-DB_PORT=5432
-# Optional ElasticSearch configuration
-# ES_ENABLED=true
-# ES_HOST=es
-# ES_PORT=9200
+# This is a sample configuration file. You can generate your configuration
+# with the `rake mastodon:setup` interactive setup wizard, but to customize
+# your setup even further, you'll need to edit it manually. This sample does
+# not demonstrate all available configuration options. Please look at
+# https://docs.joinmastodon.org/admin/config/ for the full documentation.
 
 # Federation
-# Note: Changing LOCAL_DOMAIN at a later time will cause unwanted side effects, including breaking all existing federation.
-# LOCAL_DOMAIN should *NOT* contain the protocol part of the domain e.g https://example.com.
+# ----------
+# This identifies your server and cannot be changed safely later
+# ----------
 LOCAL_DOMAIN=example.com
 
-# Changing LOCAL_HTTPS in production is no longer supported. (Mastodon will always serve https:// links)
+# Redis
+# -----
+REDIS_HOST=localhost
+REDIS_PORT=6379
 
-# Use this only if you need to run mastodon on a different domain than the one used for federation.
-# You can read more about this option on https://github.com/tootsuite/documentation/blob/master/Running-Mastodon/Serving_a_different_domain.md
-# DO *NOT* USE THIS UNLESS YOU KNOW *EXACTLY* WHAT YOU ARE DOING.
-# WEB_DOMAIN=mastodon.example.com
+# PostgreSQL
+# ----------
+DB_HOST=/var/run/postgresql
+DB_USER=mastodon
+DB_NAME=mastodon_production
+DB_PASS=
+DB_PORT=5432
 
-# Use this if you want to have several aliases handler@example1.com
-# handler@example2.com etc. for the same user. LOCAL_DOMAIN should not
-# be added. Comma separated values
-# ALTERNATE_DOMAINS=example1.com,example2.com
+# ElasticSearch (optional)
+# ------------------------
+ES_ENABLED=true
+ES_HOST=localhost
+ES_PORT=9200
 
-# Application secrets
-# Generate each with the `RAILS_ENV=production bundle exec rake secret` task (`docker-compose run --rm web rake secret` if you use docker compose)
+# Secrets
+# -------
+# Make sure to use `rake secret` to generate secrets
+# -------
 SECRET_KEY_BASE=
 OTP_SECRET=
 
-# VAPID keys (used for push notifications
-# You can generate the keys using the following command (first is the private key, second is the public one)
-# You should only generate this once per instance. If you later decide to change it, all push subscription will
-# be invalidated, requiring the users to access the website again to resubscribe.
-#
-# Generate with `RAILS_ENV=production bundle exec rake mastodon:webpush:generate_vapid_key` task (`docker-compose run --rm web rake mastodon:webpush:generate_vapid_key` if you use docker compose)
-#
-# For more information visit https://rossta.net/blog/using-the-web-push-api-with-vapid.html
+# Web Push
+# --------
+# Generate with `rake mastodon:webpush:generate_vapid_key`
+# --------
 VAPID_PRIVATE_KEY=
 VAPID_PUBLIC_KEY=
 
-# Registrations
-# Single user mode will disable registrations and redirect frontpage to the first profile
-# SINGLE_USER_MODE=true
-# Prevent registrations with following e-mail domains
-# EMAIL_DOMAIN_BLACKLIST=example1.com|example2.de|etc
-# Only allow registrations with the following e-mail domains
-# EMAIL_DOMAIN_WHITELIST=example1.com|example2.de|etc
-
-# Optionally change default language
-# DEFAULT_LOCALE=de
-
-# E-mail configuration
-# Note: Mailgun and SparkPost (https://sparkpo.st/smtp) each have good free tiers
-# If you want to use an SMTP server without authentication (e.g local Postfix relay)
-# then set SMTP_AUTH_METHOD and SMTP_OPENSSL_VERIFY_MODE to 'none' and
-# *comment* SMTP_LOGIN and SMTP_PASSWORD (leaving them blank is not enough).
+# Sending mail
+# ------------
 SMTP_SERVER=smtp.mailgun.org
 SMTP_PORT=587
 SMTP_LOGIN=
 SMTP_PASSWORD=
-SMTP_FROM_ADDRESS=notifications@example.com
-#SMTP_DOMAIN= # defaults to LOCAL_DOMAIN
-#SMTP_DELIVERY_METHOD=smtp # delivery method can also be sendmail
-#SMTP_AUTH_METHOD=plain
-#SMTP_CA_FILE=/etc/ssl/certs/ca-certificates.crt
-#SMTP_OPENSSL_VERIFY_MODE=peer
-#SMTP_ENABLE_STARTTLS_AUTO=true
-#SMTP_TLS=true
+SMTP_FROM_ADDRESS=notificatons@example.com
 
-# Optional user upload path and URL (images, avatars). Default is :rails_root/public/system. If you set this variable, you are responsible for making your HTTP server (eg. nginx) serve these files.
-# PAPERCLIP_ROOT_PATH=/var/lib/mastodon/public-system
-# PAPERCLIP_ROOT_URL=/system
-
-# Optional asset host for multi-server setups
-# The asset host must allow cross origin request from WEB_DOMAIN or LOCAL_DOMAIN
-# if WEB_DOMAIN is not set. For example, the server may have the
-# following header field:
-# Access-Control-Allow-Origin: https://example.com/
-# CDN_HOST=https://assets.example.com
-
-# S3 (optional)
-# The attachment host must allow cross origin request from WEB_DOMAIN or
-# LOCAL_DOMAIN if WEB_DOMAIN is not set. For example, the server may have the
-# following header field:
-# Access-Control-Allow-Origin: https://192.168.1.123:9000/
-# S3_ENABLED=true
-# S3_BUCKET=
-# AWS_ACCESS_KEY_ID=
-# AWS_SECRET_ACCESS_KEY=
-# S3_REGION=
-# S3_PROTOCOL=http
-# S3_HOSTNAME=192.168.1.123:9000
-
-# S3 (Minio Config (optional) Please check Minio instance for details)
-# The attachment host must allow cross origin request - see the description
-# above.
-# S3_ENABLED=true
-# S3_BUCKET=
-# AWS_ACCESS_KEY_ID=
-# AWS_SECRET_ACCESS_KEY=
-# S3_REGION=
-# S3_PROTOCOL=https
-# S3_HOSTNAME=
-# S3_ENDPOINT=
-# S3_SIGNATURE_VERSION=
-
-# Swift (optional)
-# The attachment host must allow cross origin request - see the description
-# above.
-# SWIFT_ENABLED=true
-# SWIFT_USERNAME=
-# For Keystone V3, the value for SWIFT_TENANT should be the project name
-# SWIFT_TENANT=
-# SWIFT_PASSWORD=
-# Some OpenStack V3 providers require PROJECT_ID (optional)
-# SWIFT_PROJECT_ID=
-# Keystone V2 and V3 URLs are supported. Use a V3 URL if possible to avoid
-# issues with token rate-limiting during high load.
-# SWIFT_AUTH_URL=
-# SWIFT_CONTAINER=
-# SWIFT_OBJECT_URL=
-# SWIFT_REGION=
-# Defaults to 'default'
-# SWIFT_DOMAIN_NAME=
-# Defaults to 60 seconds. Set to 0 to disable
-# SWIFT_CACHE_TTL=
-
-# Optional alias for S3 (e.g. to serve files on a custom domain, possibly using Cloudfront or Cloudflare)
-# S3_ALIAS_HOST=
-
-# Streaming API integration
-# STREAMING_API_BASE_URL=
-
-# Advanced settings
-# If you need to use pgBouncer, you need to disable prepared statements:
-# PREPARED_STATEMENTS=false
-
-# Cluster number setting for streaming API server.
-# If you comment out following line, cluster number will be `numOfCpuCores - 1`.
-STREAMING_CLUSTER_NUM=1
-
-# Docker mastodon user
-# If you use Docker, you may want to assign UID/GID manually.
-# UID=1000
-# GID=1000
-
-# LDAP authentication (optional)
-# LDAP_ENABLED=true
-# LDAP_HOST=localhost
-# LDAP_PORT=389
-# LDAP_METHOD=simple_tls
-# LDAP_BASE=
-# LDAP_BIND_DN=
-# LDAP_PASSWORD=
-# LDAP_UID=cn
-# LDAP_SEARCH_FILTER="%{uid}=%{email}"
-
-# PAM authentication (optional)
-# PAM authentication uses for the email generation the "email" pam variable
-# and optional as fallback PAM_DEFAULT_SUFFIX
-# The pam environment variable "email" is provided by:
-# https://github.com/devkral/pam_email_extractor
-# PAM_ENABLED=true
-# Fallback email domain for email address generation (LOCAL_DOMAIN by default)
-# PAM_EMAIL_DOMAIN=example.com
-# Name of the pam service (pam "auth" section is evaluated)
-# PAM_DEFAULT_SERVICE=rpam
-# Name of the pam service used for checking if an user can register (pam "account" section is evaluated) (nil (disabled) by default)
-# PAM_CONTROLLED_SERVICE=rpam
-
-# Global OAuth settings (optional) :
-# If you have only one strategy, you may want to enable this
-# OAUTH_REDIRECT_AT_SIGN_IN=true
-
-# Optional CAS authentication (cf. omniauth-cas) :
-# CAS_ENABLED=true
-# CAS_URL=https://sso.myserver.com/
-# CAS_HOST=sso.myserver.com/
-# CAS_PORT=443
-# CAS_SSL=true
-# CAS_VALIDATE_URL=
-# CAS_CALLBACK_URL=
-# CAS_LOGOUT_URL=
-# CAS_LOGIN_URL=
-# CAS_UID_FIELD='user'
-# CAS_CA_PATH=
-# CAS_DISABLE_SSL_VERIFICATION=false
-# CAS_UID_KEY='user'
-# CAS_NAME_KEY='name'
-# CAS_EMAIL_KEY='email'
-# CAS_NICKNAME_KEY='nickname'
-# CAS_FIRST_NAME_KEY='firstname'
-# CAS_LAST_NAME_KEY='lastname'
-# CAS_LOCATION_KEY='location'
-# CAS_IMAGE_KEY='image'
-# CAS_PHONE_KEY='phone'
-
-# Optional SAML authentication (cf. omniauth-saml)
-# SAML_ENABLED=true
-# SAML_ACS_URL=
-# SAML_ISSUER=http://localhost:3000/auth/auth/saml/callback
-# SAML_IDP_SSO_TARGET_URL=https://idp.testshib.org/idp/profile/SAML2/Redirect/SSO
-# SAML_IDP_CERT=
-# SAML_IDP_CERT_FINGERPRINT=
-# SAML_NAME_IDENTIFIER_FORMAT=
-# SAML_CERT=
-# SAML_PRIVATE_KEY=
-# SAML_SECURITY_WANT_ASSERTION_SIGNED=true
-# SAML_SECURITY_WANT_ASSERTION_ENCRYPTED=true
-# SAML_SECURITY_ASSUME_EMAIL_IS_VERIFIED=true
-# SAML_ATTRIBUTES_STATEMENTS_UID="urn:oid:0.9.2342.19200300.100.1.1"
-# SAML_ATTRIBUTES_STATEMENTS_EMAIL="urn:oid:1.3.6.1.4.1.5923.1.1.1.6"
-# SAML_ATTRIBUTES_STATEMENTS_FULL_NAME="urn:oid:2.16.840.1.113730.3.1.241"
-# SAML_ATTRIBUTES_STATEMENTS_FIRST_NAME="urn:oid:2.5.4.42"
-# SAML_ATTRIBUTES_STATEMENTS_LAST_NAME="urn:oid:2.5.4.4"
-# SAML_UID_ATTRIBUTE="urn:oid:0.9.2342.19200300.100.1.1"
-# SAML_ATTRIBUTES_STATEMENTS_VERIFIED=
-# SAML_ATTRIBUTES_STATEMENTS_VERIFIED_EMAIL=
-
-# Use HTTP proxy for outgoing request (optional)
-# http_proxy=http://gateway.local:8118
-# Access control for hidden service.
-# ALLOW_ACCESS_TO_HIDDEN_SERVICE=true
+# File storage (optional)
+# -----------------------
+S3_ENABLED=true
+S3_BUCKET=files.example.com
+AWS_ACCESS_KEY_ID=
+AWS_SECRET_ACCESS_KEY=
+S3_ALIAS_HOST=files.example.com

.env.test

@@ -1,5 +1,5 @@
 # Node.js
-NODE_ENV=test
+NODE_ENV=tests
 # Federation
 LOCAL_DOMAIN=cb6e6126.ngrok.io
 LOCAL_HTTPS=true

.env.vagrant

@@ -1,2 +1,4 @@
 VAGRANT=true
 LOCAL_DOMAIN=mastodon.local
+BIND=0.0.0.0
+DB_HOST=/var/run/postgresql/

.eslintignore

@@ -1,30 +1,13 @@
-# See https://help.github.com/articles/ignoring-files for more about ignoring files.
-#
-# If you find yourself ignoring temporary files generated by your text editor
-# or operating system, you probably want to add a global ignore instead:
-#   git config --global core.excludesfile '~/.gitignore_global'
-
-# Ignore bundler config.
-/.bundle
-
-# Ignore the default SQLite database.
-/db/*.sqlite3
-/db/*.sqlite3-journal
-
-# Ignore all logfiles and tempfiles.
-/log/*
-!/log/.keep
-/tmp
-coverage
-public/system
-public/assets
-.env
-.env.production
-node_modules/
-neo4j/
-
-# Ignore Vagrant files
-.vagrant/
-
-# Ignore Capistrano customizations
-config/deploy/*
+/build/**
+/coverage/**
+/db/**
+/lib/**
+/log/**
+/node_modules/**
+/nonobox/**
+/public/**
+!/public/embed.js
+/spec/**
+/tmp/**
+/vendor/**
+!.eslintrc.js

.eslintrc.js

@@ -0,0 +1,209 @@
+module.exports = {
+  root: true,
+
+  env: {
+    browser: true,
+    node: true,
+    es6: true,
+    jest: true,
+  },
+
+  globals: {
+    ATTACHMENT_HOST: false,
+  },
+
+  parser: 'babel-eslint',
+
+  plugins: [
+    'react',
+    'jsx-a11y',
+    'import',
+    'promise',
+  ],
+
+  parserOptions: {
+    sourceType: 'module',
+    ecmaFeatures: {
+      experimentalObjectRestSpread: true,
+      jsx: true,
+    },
+    ecmaVersion: 2018,
+  },
+
+  settings: {
+    react: {
+      version: 'detect',
+    },
+    'import/extensions': [
+      '.js',
+    ],
+    'import/ignore': [
+      'node_modules',
+      '\\.(css|scss|json)$',
+    ],
+    'import/resolver': {
+      node: {
+        paths: ['app/javascript'],
+      },
+    },
+  },
+
+  rules: {
+    'brace-style': 'warn',
+    'comma-dangle': ['error', 'always-multiline'],
+    'comma-spacing': [
+      'warn',
+      {
+        before: false,
+        after: true,
+      },
+    ],
+    'comma-style': ['warn', 'last'],
+    'consistent-return': 'error',
+    'dot-notation': 'error',
+    eqeqeq: 'error',
+    indent: ['warn', 2],
+    'jsx-quotes': ['error', 'prefer-single'],
+    'no-catch-shadow': 'error',
+    'no-cond-assign': 'error',
+    'no-console': [
+      'warn',
+      {
+        allow: [
+          'error',
+          'warn',
+        ],
+      },
+    ],
+    'no-fallthrough': 'error',
+    'no-irregular-whitespace': 'error',
+    'no-mixed-spaces-and-tabs': 'warn',
+    'no-nested-ternary': 'warn',
+    'no-trailing-spaces': 'warn',
+    'no-undef': 'error',
+    'no-unreachable': 'error',
+    'no-unused-expressions': 'error',
+    'no-unused-vars': [
+      'error',
+      {
+        vars: 'all',
+        args: 'after-used',
+        ignoreRestSiblings: true,
+      },
+    ],
+    'object-curly-spacing': ['error', 'always'],
+    'padded-blocks': [
+      'error',
+      {
+        classes: 'always',
+      },
+    ],
+    quotes: ['error', 'single'],
+    semi: 'error',
+    strict: 'off',
+    'valid-typeof': 'error',
+
+    'react/jsx-boolean-value': 'error',
+    'react/jsx-closing-bracket-location': ['error', 'line-aligned'],
+    'react/jsx-curly-spacing': 'error',
+    'react/jsx-equals-spacing': 'error',
+    'react/jsx-first-prop-new-line': ['error', 'multiline-multiprop'],
+    'react/jsx-indent': ['error', 2],
+    'react/jsx-no-bind': 'error',
+    'react/jsx-no-duplicate-props': 'error',
+    'react/jsx-no-undef': 'error',
+    'react/jsx-tag-spacing': 'error',
+    'react/jsx-uses-react': 'error',
+    'react/jsx-uses-vars': 'error',
+    'react/jsx-wrap-multilines': 'error',
+    'react/no-multi-comp': 'off',
+    'react/no-string-refs': 'error',
+    'react/prop-types': 'error',
+    'react/self-closing-comp': 'error',
+
+    'jsx-a11y/accessible-emoji': 'warn',
+    'jsx-a11y/alt-text': 'warn',
+    'jsx-a11y/anchor-has-content': 'warn',
+    'jsx-a11y/anchor-is-valid': [
+      'warn',
+      {
+        components: [
+          'Link',
+          'NavLink',
+        ],
+        specialLink: [
+          'to',
+        ],
+        aspect: [
+          'noHref',
+          'invalidHref',
+          'preferButton',
+        ],
+      },
+    ],
+    'jsx-a11y/aria-activedescendant-has-tabindex': 'warn',
+    'jsx-a11y/aria-props': 'warn',
+    'jsx-a11y/aria-proptypes': 'warn',
+    'jsx-a11y/aria-role': 'warn',
+    'jsx-a11y/aria-unsupported-elements': 'warn',
+    'jsx-a11y/heading-has-content': 'warn',
+    'jsx-a11y/html-has-lang': 'warn',
+    'jsx-a11y/iframe-has-title': 'warn',
+    'jsx-a11y/img-redundant-alt': 'warn',
+    'jsx-a11y/interactive-supports-focus': 'warn',
+    'jsx-a11y/label-has-for': 'off',
+    'jsx-a11y/mouse-events-have-key-events': 'warn',
+    'jsx-a11y/no-access-key': 'warn',
+    'jsx-a11y/no-distracting-elements': 'warn',
+    'jsx-a11y/no-noninteractive-element-interactions': [
+      'warn',
+      {
+        handlers: [
+          'onClick',
+        ],
+      },
+    ],
+    'jsx-a11y/no-onchange': 'warn',
+    'jsx-a11y/no-redundant-roles': 'warn',
+    'jsx-a11y/no-static-element-interactions': [
+      'warn',
+      {
+        handlers: [
+          'onClick',
+        ],
+      },
+    ],
+    'jsx-a11y/role-has-required-aria-props': 'warn',
+    'jsx-a11y/role-supports-aria-props': 'off',
+    'jsx-a11y/scope': 'warn',
+    'jsx-a11y/tabindex-no-positive': 'warn',
+
+    'import/extensions': [
+      'error',
+      'always',
+      {
+        js: 'never',
+      },
+    ],
+    'import/newline-after-import': 'error',
+    'import/no-extraneous-dependencies': [
+      'error',
+      {
+        devDependencies: [
+          'config/webpack/**',
+          'app/javascript/mastodon/test_setup.js',
+          'app/javascript/**/__tests__/**',
+        ],
+      },
+    ],
+    'import/no-unresolved': 'error',
+    'import/no-webpack-loader-syntax': 'error',
+
+    'promise/catch-or-return': [
+      'error',
+      {
+        allowFinally: true,
+      },
+    ],
+  },
+};

.eslintrc.yml

@@ -1,170 +0,0 @@
----
-root: true
-
-env:
-  browser: true
-  node: true
-  es6: true
-  jest: true
-
-globals:
-  ATTACHMENT_HOST: false
-
-parser: babel-eslint
-
-plugins:
-- react
-- jsx-a11y
-- import
-- promise
-
-parserOptions:
-  sourceType: module
-  ecmaFeatures:
-    experimentalObjectRestSpread: true
-    jsx: true
-  ecmaVersion: 2018
-
-settings:
-  import/extensions:
-  - .js
-  import/ignore:
-  - node_modules
-  - \\.(css|scss|json)$
-
-rules:
-  brace-style: warn
-  comma-dangle:
-  - error
-  - always-multiline
-  comma-spacing:
-  - warn
-  - before: false
-    after: true
-  comma-style:
-  - warn
-  - last
-  consistent-return: error
-  dot-notation: error
-  eqeqeq: error
-  indent:
-  - warn
-  - 2
-  jsx-quotes:
-  - error
-  - prefer-single
-  no-catch-shadow: error
-  no-cond-assign: error
-  no-console:
-  - warn
-  - allow:
-    - error
-    - warn
-  no-fallthrough: error
-  no-irregular-whitespace: error
-  no-mixed-spaces-and-tabs: warn
-  no-nested-ternary: warn
-  no-trailing-spaces: warn
-  no-undef: error
-  no-unreachable: error
-  no-unused-expressions: error
-  no-unused-vars:
-  - error
-  - vars: all
-    args: after-used
-    ignoreRestSiblings: true
-  object-curly-spacing:
-  - error
-  - always
-  padded-blocks:
-  - error
-  - classes: always
-  quotes:
-  - error
-  - single
-  semi: error
-  strict: off
-  valid-typeof: error
-
-  react/jsx-boolean-value: error
-  react/jsx-closing-bracket-location:
-  - error
-  - line-aligned
-  react/jsx-curly-spacing: error
-  react/jsx-equals-spacing: error
-  react/jsx-first-prop-new-line:
-  - error
-  - multiline-multiprop
-  react/jsx-indent:
-  - error
-  - 2
-  react/jsx-no-bind: error
-  react/jsx-no-duplicate-props: error
-  react/jsx-no-undef: error
-  react/jsx-tag-spacing: error
-  react/jsx-uses-react: error
-  react/jsx-uses-vars: error
-  react/jsx-wrap-multilines: error
-  react/no-multi-comp: off
-  react/no-string-refs: error
-  react/prop-types: error
-  react/self-closing-comp: error
-
-  jsx-a11y/accessible-emoji: warn
-  jsx-a11y/alt-text: warn
-  jsx-a11y/anchor-has-content: warn
-  jsx-a11y/anchor-is-valid:
-  - warn
-  - components:
-    - Link
-    - NavLink
-    specialLink:
-    - to
-    aspect:
-    - noHref
-    - invalidHref
-    - preferButton
-  jsx-a11y/aria-activedescendant-has-tabindex: warn
-  jsx-a11y/aria-props: warn
-  jsx-a11y/aria-proptypes: warn
-  jsx-a11y/aria-role: warn
-  jsx-a11y/aria-unsupported-elements: warn
-  jsx-a11y/heading-has-content: warn
-  jsx-a11y/html-has-lang: warn
-  jsx-a11y/iframe-has-title: warn
-  jsx-a11y/img-redundant-alt: warn
-  jsx-a11y/interactive-supports-focus: warn
-  jsx-a11y/label-has-for: off
-  jsx-a11y/mouse-events-have-key-events: warn
-  jsx-a11y/no-access-key: warn
-  jsx-a11y/no-distracting-elements: warn
-  jsx-a11y/no-noninteractive-element-interactions:
-  - warn
-  - handlers:
-    - onClick
-  jsx-a11y/no-onchange: warn
-  jsx-a11y/no-redundant-roles: warn
-  jsx-a11y/no-static-element-interactions:
-  - warn
-  - handlers:
-    - onClick
-  jsx-a11y/role-has-required-aria-props: warn
-  jsx-a11y/role-supports-aria-props: off
-  jsx-a11y/scope: warn
-  jsx-a11y/tabindex-no-positive: warn
-
-  import/extensions:
-  - error
-  - always
-  - js: never
-  import/newline-after-import: error
-  import/no-extraneous-dependencies:
-  - error
-  - devDependencies:
-    - "config/webpack/**"
-    - "app/javascript/mastodon/test_setup.js"
-    - "app/javascript/**/__tests__/**"
-  import/no-unresolved: error
-  import/no-webpack-loader-syntax: error
-
-  promise/catch-or-return: error

.github/FUNDING.yml

@@ -0,0 +1,3 @@
+patreon: mastodon
+open_collective: mastodon
+github: [Gargron]

.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1,5 @@
+blank_issues_enabled: false
+contact_links:
+  - name: Mastodon Meta Discussion Board
+    url: https://discourse.joinmastodon.org/
+    about: Please ask and answer questions here.

.github/dependabot.yml

@@ -0,0 +1,22 @@
+# To get started with Dependabot version updates, you'll need to specify which
+# package ecosystems to update and where the package manifests are located.
+# Please see the documentation for all configuration options:
+# https://help.github.com/github/administering-a-repository/configuration-options-for-dependency-updates
+
+version: 2
+updates:
+  - package-ecosystem: npm
+    directory: "/"
+    schedule:
+      interval: weekly
+    open-pull-requests-limit: 99
+    allow:
+      - dependency-type: all
+
+  - package-ecosystem: bundler
+    directory: "/"
+    schedule:
+      interval: weekly
+    open-pull-requests-limit: 99
+    allow:
+      - dependency-type: all

.github/stale.yml

@@ -0,0 +1,10 @@
+daysUntilStale: 120
+daysUntilClose: 7
+exemptLabels:
+  - security
+staleLabel: wontfix
+markComment: >
+  This issue has been automatically marked as stale because it has not had
+  recent activity. It will be closed if no further activity occurs. Thank you
+  for your contributions.
+only: pulls

.gitignore

@@ -13,33 +13,40 @@
 /db/*.sqlite3-journal
 
 # Ignore all logfiles and tempfiles.
+.eslintcache
 /log/*
 !/log/.keep
 /tmp
-coverage
-public/system
-public/assets
-public/packs
-public/packs-test
+/coverage
+/public/system
+/public/assets
+/public/packs
+/public/packs-test
 .env
 .env.production
-node_modules/
-build/
+.env.development
+/node_modules/
+/build/
 
 # Ignore Vagrant files
 .vagrant/
 
 # Ignore Capistrano customizations
-config/deploy/*
+/config/deploy/*
 
 # Ignore IDE files
 .vscode/
 .idea/
 
 # Ignore postgres + redis + elasticsearch volume optionally created by docker-compose
-postgres
-redis
-elasticsearch
+/postgres
+/redis
+/elasticsearch
+
+# ignore Helm lockfile, dependency charts, and local values file
+/chart/Chart.lock
+/chart/charts/*.tgz
+/chart/values.yaml
 
 # Ignore Apple files
 .DS_Store
@@ -55,6 +62,8 @@ npm-debug.log
 yarn-error.log
 yarn-debug.log
 
+# Ignore vagrant log files
+*-cloudimg-console.log
+
 # Ignore Docker option files
 docker-compose.override.yml
-

.nvmrc

@@ -1 +1 @@
-8
+12

.postcssrc.yml

@@ -1,9 +0,0 @@
-plugins:
-  postcss-smart-import: {}
-  precss: {}
-  autoprefixer:
-    browsers:
-      - last 2 versions
-      - IE >= 11
-      - iOS >= 9
-  postcss-object-fit-images: {}

.rubocop.yml

@@ -1,5 +1,8 @@
+require:
+  - rubocop-rails
+
 AllCops:
-  TargetRubyVersion: 2.3
+  TargetRubyVersion: 2.4
   Exclude:
   - 'spec/**/*'
   - 'db/**/*'
@@ -25,6 +28,10 @@ Layout/EmptyLineAfterMagicComment:
 Layout/SpaceInsideHashLiteralBraces:
   EnforcedStyle: space
 
+Lint/UselessAccessModifier:
+  ContextCreatingMethods:
+    - class_methods
+
 Metrics/AbcSize:
   Max: 100
 
@@ -43,7 +50,7 @@ Metrics/ClassLength:
 Metrics/CyclomaticComplexity:
   Max: 25
 
-Metrics/LineLength:
+Layout/LineLength:
   AllowURI: true
   Enabled: false
 
@@ -68,6 +75,9 @@ Naming/MemoizedInstanceVariableName:
 Rails:
   Enabled: true
 
+Rails/EnumHash:
+  Enabled: false
+
 Rails/HasAndBelongsToMany:
   Enabled: false
 
@@ -80,7 +90,10 @@ Rails/HttpStatus:
 Rails/Exit:
   Exclude:
     - 'lib/mastodon/*'
-    - 'lib/cli'
+    - 'lib/cli.rb'
+
+Rails/HelperInstanceVariable:
+  Enabled: false
 
 Style/ClassAndModuleChildren:
   Enabled: false
@@ -96,6 +109,9 @@ Style/Documentation:
 Style/DoubleNegation:
   Enabled: true
 
+Style/FormatStringToken:
+  Enabled: false
+
 Style/FrozenStringLiteralComment:
   Enabled: true
 

.ruby-version

@@ -1 +1 @@
-2.5.3
+2.6.6

.sass-lint.yml

@@ -0,0 +1,37 @@
+# Linter Documentation:
+# https://github.com/sasstools/sass-lint/tree/v1.13.1/docs/options
+
+files:
+  include: app/javascript/styles/**/*.scss
+  ignore:
+    - app/javascript/styles/mastodon/reset.scss
+
+rules:
+  # Disallows
+  no-color-literals: 0
+  no-css-comments: 0
+  no-duplicate-properties: 0
+  no-ids: 0
+  no-important: 0
+  no-mergeable-selectors: 0
+  no-misspelled-properties: 0
+  no-qualifying-elements: 0
+  no-transition-all: 0
+  no-vendor-prefixes: 0
+
+  # Nesting
+  force-element-nesting: 0
+  force-attribute-nesting: 0
+  force-pseudo-nesting: 0
+
+  # Name Formats
+  class-name-format: 0
+  leading-zero: 0
+
+  # Style Guide
+  attribute-quotes: 0
+  hex-length: 0
+  indentation: 0
+  nesting-depth: 0
+  property-sort-order: 0
+  quotes: 0

.scss-lint.yml

@@ -1,264 +0,0 @@
-# Linter Documentation:
-# https://github.com/brigade/scss-lint/blob/v0.42.2/lib/scss_lint/linter/README.md
-
-scss_files: 'app/javascript/styles/**/*.scss'
-
-exclude:
-  - app/javascript/styles/reset.scss
-
-linters:
-  # Reports when you use improper spacing around ! (the "bang") in !default,
-  # !global, !important, and !optional flags.
-  BangFormat:
-    enabled: false
-
-  # Whether or not to prefer `border: 0` over `border: none`.
-  BorderZero:
-    enabled: false
-
-  # Reports when you define a rule set using a selector with chained classes
-  # (a.k.a. adjoining classes).
-  ChainedClasses:
-    enabled: false
-
-  # Prefer hexadecimal color codes over color keywords.
-  # (e.g. `color: green` is a color keyword)
-  ColorKeyword:
-    enabled: false
-
-  # Prefer color literals (keywords or hexadecimal codes) to be used only in
-  # variable declarations. They should be referred to via variables everywhere
-  # else.
-  ColorVariable:
-    enabled: true
-
-  # Which form of comments to prefer in CSS.
-  Comment:
-    enabled: false
-
-  # Reports @debug statements (which you probably left behind accidentally).
-  DebugStatement:
-    enabled: false
-
-  # Rule sets should be ordered as follows:
-  # - @extend declarations
-  # - @include declarations without inner @content
-  # - properties, @include declarations with inner @content
-  # - nested rule sets.
-  DeclarationOrder:
-    enabled: false
-
-  # `scss-lint:disable` control comments should be preceded by a comment
-  # explaining why these linters are being disabled for this file.
-  # See https://github.com/brigade/scss-lint#disabling-linters-via-source for
-  # more information.
-  DisableLinterReason:
-    enabled: true
-
-  # Reports when you define the same property twice in a single rule set.
-  DuplicateProperty:
-    enabled: false
-
-  # Separate rule, function, and mixin declarations with empty lines.
-  EmptyLineBetweenBlocks:
-    enabled: true
-
-  # Reports when you have an empty rule set.
-  EmptyRule:
-    enabled: true
-
-  # Reports when you have an @extend directive.
-  ExtendDirective:
-    enabled: false
-
-  # Files should always have a final newline. This results in better diffs
-  # when adding lines to the file, since SCM systems such as git won't
-  # think that you touched the last line.
-  FinalNewline:
-    enabled: false
-
-  # HEX colors should use three-character values where possible.
-  HexLength:
-    enabled: false
-
-  # HEX color values should use lower-case colors to differentiate between
-  # letters and numbers, e.g. `#E3E3E3` vs. `#e3e3e3`.
-  HexNotation:
-    enabled: true
-
-  # Avoid using ID selectors.
-  IdSelector:
-    enabled: false
-
-  # The basenames of @imported SCSS partials should not begin with an
-  # underscore and should not include the filename extension.
-  ImportPath:
-    enabled: false
-
-  # Avoid using !important in properties. It is usually indicative of a
-  # misunderstanding of CSS specificity and can lead to brittle code.
-  ImportantRule:
-    enabled: false
-
-  # Indentation should always be done in increments of 2 spaces.
-  Indentation:
-    enabled: true
-    width: 2
-
-  # Don't write leading zeros for numeric values with a decimal point.
-  LeadingZero:
-    enabled: false
-
-  # Reports when you define the same selector twice in a single sheet.
-  MergeableSelector:
-    enabled: false
-
-  # Functions, mixins, variables, and placeholders should be declared
-  # with all lowercase letters and hyphens instead of underscores.
-  NameFormat:
-    enabled: false
-
-  # Avoid nesting selectors too deeply.
-  NestingDepth:
-    enabled: false
-
-  # Always use placeholder selectors in @extend.
-  PlaceholderInExtend:
-    enabled: false
-
-  # Sort properties in a strict order.
-  PropertySortOrder:
-    enabled: false
-
-  # Reports when you use an unknown or disabled CSS property
-  # (ignoring vendor-prefixed properties).
-  PropertySpelling:
-    enabled: false
-
-  # Configure which units are allowed for property values.
-  PropertyUnits:
-    enabled: false
-
-  # Pseudo-elements, like ::before, and ::first-letter, should be declared
-  # with two colons. Pseudo-classes, like :hover and :first-child, should
-  # be declared with one colon.
-  PseudoElement:
-    enabled: true
-
-  # Avoid qualifying elements in selectors (also known as "tag-qualifying").
-  QualifyingElement:
-    enabled: false
-
-  # Don't write selectors with a depth of applicability greater than 3.
-  SelectorDepth:
-    enabled: false
-
-  # Selectors should always use hyphenated-lowercase, rather than camelCase or
-  # snake_case.
-  SelectorFormat:
-    enabled: false
-    convention: hyphenated_lowercase
-
-  # Prefer the shortest shorthand form possible for properties that support it.
-  Shorthand:
-    enabled: true
-
-  # Each property should have its own line, except in the special case of
-  # single line rulesets.
-  SingleLinePerProperty:
-    enabled: true
-    allow_single_line_rule_sets: true
-
-  # Split selectors onto separate lines after each comma, and have each
-  # individual selector occupy a single line.
-  SingleLinePerSelector:
-    enabled: true
-
-  # Commas in lists should be followed by a space.
-  SpaceAfterComma:
-    enabled: false
-
-  # Properties should be formatted with a single space separating the colon
-  # from the property's value.
-  SpaceAfterPropertyColon:
-    enabled: true
-
-  # Properties should be formatted with no space between the name and the
-  # colon.
-  SpaceAfterPropertyName:
-    enabled: true
-
-  # Variables should be formatted with a single space separating the colon
-  # from the variable's value.
-  SpaceAfterVariableColon:
-    enabled: true
-
-  # Variables should be formatted with no space between the name and the
-  # colon.
-  SpaceAfterVariableName:
-    enabled: false
-
-  # Operators should be formatted with a single space on both sides of an
-  # infix operator.
-  SpaceAroundOperator:
-    enabled: true
-
-  # Opening braces should be preceded by a single space.
-  SpaceBeforeBrace:
-    enabled: true
-
-  # Parentheses should not be padded with spaces.
-  SpaceBetweenParens:
-    enabled: false
-
-  # Enforces that string literals should be written with a consistent form
-  # of quotes (single or double).
-  StringQuotes:
-    enabled: false
-
-  # Property values, @extend, @include, and @import directives, and variable
-  # declarations should always end with a semicolon.
-  TrailingSemicolon:
-    enabled: true
-
-  # Reports lines containing trailing whitespace.
-  TrailingWhitespace:
-    enabled: true
-
-  # Don't write trailing zeros for numeric values with a decimal point.
-  TrailingZero:
-    enabled: false
-
-  # Don't use the `all` keyword to specify transition properties.
-  TransitionAll:
-    enabled: false
-
-  # Numeric values should not contain unnecessary fractional portions.
-  UnnecessaryMantissa:
-    enabled: false
-
-  # Do not use parent selector references (&) when they would otherwise
-  # be unnecessary.
-  UnnecessaryParentReference:
-    enabled: false
-
-  # URLs should be valid and not contain protocols or domain names.
-  UrlFormat:
-    enabled: true
-
-  # URLs should always be enclosed within quotes.
-  UrlQuotes:
-    enabled: true
-
-  # Properties, like color and font, are easier to read and maintain
-  # when defined using variables rather than literals.
-  VariableForProperty:
-    enabled: false
-
-  # Avoid vendor prefixes. Or rather: don't write them yourself.
-  VendorPrefix:
-    enabled: false
-
-  # Omit length units on zero values, e.g. `0px` vs. `0`.
-  ZeroUnit:
-    enabled: true

.yarnclean

@@ -43,4 +43,4 @@ Gruntfile.js
 
 # for specific ignore
 !.svgo.yml
-
+!sass-lint/**/*.yml

CONTRIBUTING.md

@@ -1,7 +1,7 @@
 Contributing
 ============
 
-Thank you for considering contributing to Mastodon 🐘 
+Thank you for considering contributing to Mastodon 🐘
 
 You can contribute in the following ways:
 
@@ -10,15 +10,17 @@ You can contribute in the following ways:
 - Contributing code to Mastodon by fixing bugs or implementing features
 - Improving the documentation
 
+If your contributions are accepted into Mastodon, you can request to be paid through [our OpenCollective](https://opencollective.com/mastodon).
+
 ## Bug reports
 
-Bug reports and feature suggestions can be submitted to [GitHub Issues](https://github.com/tootsuite/mastodon/issues). Please make sure that you are not submitting duplicates, and that a similar report or request has not already been resolved or rejected in the past using the search function. Please also use descriptive, concise titles.
+Bug reports and feature suggestions must use descriptive and concise titles and be submitted to [GitHub Issues](https://github.com/tootsuite/mastodon/issues). Please use the search function to make sure that you are not submitting duplicates, and that a similar report or request has not already been resolved or rejected.
 
 ## Translations
 
-You can submit translations via [Weblate](https://weblate.joinmastodon.org/). They are periodically merged into the codebase.
+You can submit translations via [Crowdin](https://crowdin.com/project/mastodon). They are periodically merged into the codebase.
 
-[![Mastodon translation statistics by language](https://weblate.joinmastodon.org/widgets/mastodon/-/multi-auto.svg)](https://weblate.joinmastodon.org/)
+[![Crowdin](https://d322cqt584bo4o.cloudfront.net/mastodon/localized.svg)](https://crowdin.com/project/mastodon)
 
 ## Pull requests
 

Dockerfile

@@ -1,86 +1,139 @@
-FROM node:8.12.0-alpine as node
-FROM ruby:2.4.5-alpine3.8
+FROM ubuntu:20.04 as build-dep
 
-LABEL maintainer="https://github.com/tootsuite/mastodon" \
-      description="Your self-hosted, globally interconnected microblogging community"
+# Use bash for the shell
+SHELL ["bash", "-c"]
 
+# Install Node v12 (LTS)
+ENV NODE_VER="12.16.3"
+RUN ARCH= && \
+    dpkgArch="$(dpkg --print-architecture)" && \
+  case "${dpkgArch##*-}" in \
+    amd64) ARCH='x64';; \
+    ppc64el) ARCH='ppc64le';; \
+    s390x) ARCH='s390x';; \
+    arm64) ARCH='arm64';; \
+    armhf) ARCH='armv7l';; \
+    i386) ARCH='x86';; \
+    *) echo "unsupported architecture"; exit 1 ;; \
+  esac && \
+    echo "Etc/UTC" > /etc/localtime && \
+	apt update && \
+	apt -y install wget python && \
+	cd ~ && \
+	wget https://nodejs.org/download/release/v$NODE_VER/node-v$NODE_VER-linux-$ARCH.tar.gz && \
+	tar xf node-v$NODE_VER-linux-$ARCH.tar.gz && \
+	rm node-v$NODE_VER-linux-$ARCH.tar.gz && \
+	mv node-v$NODE_VER-linux-$ARCH /opt/node
+
+# Install jemalloc
+ENV JE_VER="5.2.1"
+RUN apt update && \
+	apt -y install make autoconf gcc g++ && \
+	cd ~ && \
+	wget https://github.com/jemalloc/jemalloc/archive/$JE_VER.tar.gz && \
+	tar xf $JE_VER.tar.gz && \
+	cd jemalloc-$JE_VER && \
+	./autogen.sh && \
+	./configure --prefix=/opt/jemalloc && \
+	make -j$(nproc) > /dev/null && \
+	make install_bin install_include install_lib
+
+# Install Ruby
+ENV RUBY_VER="2.6.6"
+ENV CPPFLAGS="-I/opt/jemalloc/include"
+ENV LDFLAGS="-L/opt/jemalloc/lib/"
+RUN apt update && \
+	apt -y install build-essential \
+		bison libyaml-dev libgdbm-dev libreadline-dev \
+		libncurses5-dev libffi-dev zlib1g-dev libssl-dev && \
+	cd ~ && \
+	wget https://cache.ruby-lang.org/pub/ruby/${RUBY_VER%.*}/ruby-$RUBY_VER.tar.gz && \
+	tar xf ruby-$RUBY_VER.tar.gz && \
+	cd ruby-$RUBY_VER && \
+	./configure --prefix=/opt/ruby \
+	  --with-jemalloc \
+	  --with-shared \
+	  --disable-install-doc && \
+	ln -s /opt/jemalloc/lib/* /usr/lib/ && \
+	make -j$(nproc) > /dev/null && \
+	make install
+
+ENV PATH="${PATH}:/opt/ruby/bin:/opt/node/bin"
+
+RUN npm install -g yarn && \
+	gem install bundler && \
+	apt update && \
+	apt -y install git libicu-dev libidn11-dev \
+	libpq-dev libprotobuf-dev protobuf-compiler
+
+COPY Gemfile* package.json yarn.lock /opt/mastodon/
+
+RUN cd /opt/mastodon && \
+  bundle config set deployment 'true' && \
+  bundle config set without 'development test' && \
+	bundle install -j$(nproc) && \
+	yarn install --pure-lockfile
+
+FROM ubuntu:20.04
+
+# Copy over all the langs needed for runtime
+COPY --from=build-dep /opt/node /opt/node
+COPY --from=build-dep /opt/ruby /opt/ruby
+COPY --from=build-dep /opt/jemalloc /opt/jemalloc
+
+# Add more PATHs to the PATH
+ENV PATH="${PATH}:/opt/ruby/bin:/opt/node/bin:/opt/mastodon/bin"
+
+# Create the mastodon user
 ARG UID=991
 ARG GID=991
+RUN apt update && \
+	echo "Etc/UTC" > /etc/localtime && \
+	ln -s /opt/jemalloc/lib/* /usr/lib/ && \
+	apt install -y whois wget && \
+	addgroup --gid $GID mastodon && \
+	useradd -m -u $UID -g $GID -d /opt/mastodon mastodon && \
+	echo "mastodon:`head /dev/urandom | tr -dc A-Za-z0-9 | head -c 24 | mkpasswd -s -m sha-256`" | chpasswd
 
-ENV PATH=/mastodon/bin:$PATH \
-    RAILS_SERVE_STATIC_FILES=true \
-    RAILS_ENV=production \
-    NODE_ENV=production
+# Install mastodon runtime deps
+RUN apt -y --no-install-recommends install \
+	  libssl1.1 libpq5 imagemagick ffmpeg \
+	  libicu66 libprotobuf17 libidn11 libyaml-0-2 \
+	  file ca-certificates tzdata libreadline8 && \
+	apt -y install gcc && \
+	ln -s /opt/mastodon /mastodon && \
+	gem install bundler && \
+	rm -rf /var/cache && \
+	rm -rf /var/lib/apt/lists/*
 
-ARG LIBICONV_VERSION=1.15
-ARG LIBICONV_DOWNLOAD_SHA256=ccf536620a45458d26ba83887a983b96827001e92a13847b45e4925cc8913178
+# Add tini
+ENV TINI_VERSION="0.18.0"
+ENV TINI_SUM="12d20136605531b09a2c2dac02ccee85e1b874eb322ef6baf7561cd93f93c855"
+ADD https://github.com/krallin/tini/releases/download/v${TINI_VERSION}/tini /tini
+RUN echo "$TINI_SUM tini" | sha256sum -c -
+RUN chmod +x /tini
 
-EXPOSE 3000 4000
+# Copy over mastodon source, and dependencies from building, and set permissions
+COPY --chown=mastodon:mastodon . /opt/mastodon
+COPY --from=build-dep --chown=mastodon:mastodon /opt/mastodon /opt/mastodon
 
-WORKDIR /mastodon
+# Run mastodon services in prod mode
+ENV RAILS_ENV="production"
+ENV NODE_ENV="production"
 
-COPY --from=node /usr/local/bin/node /usr/local/bin/node
-COPY --from=node /usr/local/lib/node_modules /usr/local/lib/node_modules
-COPY --from=node /usr/local/bin/npm /usr/local/bin/npm
-COPY --from=node /opt/yarn-* /opt/yarn
-
-RUN apk -U upgrade \
- && apk add -t build-dependencies \
-    build-base \
-    icu-dev \
-    libidn-dev \
-    libressl \
-    libtool \
-    postgresql-dev \
-    protobuf-dev \
-    python \
- && apk add \
-    ca-certificates \
-    ffmpeg \
-    file \
-    git \
-    icu-libs \
-    imagemagick \
-    libidn \
-    libpq \
-    protobuf \
-    tini \
-    tzdata \
- && update-ca-certificates \
- && ln -s /opt/yarn/bin/yarn /usr/local/bin/yarn \
- && ln -s /opt/yarn/bin/yarnpkg /usr/local/bin/yarnpkg \
- && mkdir -p /tmp/src /opt \
- && wget -O libiconv.tar.gz "https://ftp.gnu.org/pub/gnu/libiconv/libiconv-$LIBICONV_VERSION.tar.gz" \
- && echo "$LIBICONV_DOWNLOAD_SHA256 *libiconv.tar.gz" | sha256sum -c - \
- && tar -xzf libiconv.tar.gz -C /tmp/src \
- && rm libiconv.tar.gz \
- && cd /tmp/src/libiconv-$LIBICONV_VERSION \
- && ./configure --prefix=/usr/local \
- && make -j$(getconf _NPROCESSORS_ONLN)\
- && make install \
- && libtool --finish /usr/local/lib \
- && cd /mastodon \
- && rm -rf /tmp/* /var/cache/apk/*
-
-COPY Gemfile Gemfile.lock package.json yarn.lock .yarnclean /mastodon/
-
-RUN bundle config build.nokogiri --with-iconv-lib=/usr/local/lib --with-iconv-include=/usr/local/include \
- && bundle install -j$(getconf _NPROCESSORS_ONLN) --deployment --without test development \
- && yarn install --pure-lockfile --ignore-engines \
- && yarn cache clean
-
-RUN addgroup -g ${GID} mastodon && adduser -h /mastodon -s /bin/sh -D -G mastodon -u ${UID} mastodon \
- && mkdir -p /mastodon/public/system /mastodon/public/assets /mastodon/public/packs \
- && chown -R mastodon:mastodon /mastodon/public
-
-COPY . /mastodon
-
-RUN chown -R mastodon:mastodon /mastodon
-
-VOLUME /mastodon/public/system
+# Tell rails to serve static files
+ENV RAILS_SERVE_STATIC_FILES="true"
+ENV BIND="0.0.0.0"
 
+# Set the run user
 USER mastodon
 
-RUN OTP_SECRET=precompile_placeholder SECRET_KEY_BASE=precompile_placeholder bundle exec rails assets:precompile
+# Precompile assets
+RUN cd ~ && \
+	OTP_SECRET=precompile_placeholder SECRET_KEY_BASE=precompile_placeholder rails assets:precompile && \
+	yarn cache clean
 
-ENTRYPOINT ["/sbin/tini", "--"]
+# Set the work dir and the container entry point
+WORKDIR /opt/mastodon
+ENTRYPOINT ["/tini", "--"]
+EXPOSE 3000 4000

Gemfile

@@ -1,104 +1,116 @@
 # frozen_string_literal: true
 
 source 'https://rubygems.org'
-ruby '>= 2.3.0', '< 2.6.0'
+ruby '>= 2.5.0', '< 3.0.0'
 
-gem 'pkg-config', '~> 1.3'
+gem 'pkg-config', '~> 1.4'
 
-gem 'puma', '~> 3.12'
-gem 'rails', '~> 5.2.1'
+gem 'puma', '~> 4.3'
+gem 'rails', '~> 5.2.4.3'
+gem 'sprockets', '~> 3.7.2'
 gem 'thor', '~> 0.20'
+gem 'rack', '~> 2.2.3'
+
+gem 'thwait', '~> 0.1.0'
+gem 'e2mmap', '~> 0.1.0'
 
 gem 'hamlit-rails', '~> 0.2'
-gem 'pg', '~> 1.1'
+gem 'pg', '~> 1.2'
 gem 'makara', '~> 0.4'
-gem 'pghero', '~> 2.2'
-gem 'dotenv-rails', '~> 2.5'
+gem 'pghero', '~> 2.5'
+gem 'dotenv-rails', '~> 2.7'
 
-gem 'aws-sdk-s3', '~> 1.23', require: false
+gem 'aws-sdk-s3', '~> 1.73', require: false
 gem 'fog-core', '<= 2.1.0'
 gem 'fog-openstack', '~> 0.3', require: false
 gem 'paperclip', '~> 6.0'
 gem 'paperclip-av-transcoder', '~> 0.6'
 gem 'streamio-ffmpeg', '~> 3.0'
+gem 'blurhash', '~> 0.1'
 
 gem 'active_model_serializers', '~> 0.10'
-gem 'addressable', '~> 2.5'
-gem 'bootsnap', '~> 1.3', require: false
+gem 'addressable', '~> 2.7'
+gem 'bootsnap', '~> 1.4', require: false
 gem 'browser'
-gem 'charlock_holmes', '~> 0.7.6'
+gem 'charlock_holmes', '~> 0.7.7'
 gem 'iso-639'
-gem 'chewy', '~> 5.0'
-gem 'cld3', '~> 3.2.0'
-gem 'devise', '~> 4.5'
-gem 'devise-two-factor', '~> 3.0'
+gem 'chewy', '~> 5.1'
+gem 'cld3', '~> 3.3.0'
+gem 'devise', '~> 4.7'
+gem 'devise-two-factor', '~> 3.1'
 
 group :pam_authentication, optional: true do
   gem 'devise_pam_authenticatable2', '~> 9.2'
 end
 
-gem 'net-ldap', '~> 0.10'
+gem 'net-ldap', '~> 0.16'
 gem 'omniauth-cas', '~> 1.1'
 gem 'omniauth-saml', '~> 1.10'
-gem 'omniauth', '~> 1.2'
+gem 'omniauth', '~> 1.9'
 
-gem 'doorkeeper', '~> 5.0'
+gem 'color_diff', '~> 0.1'
+gem 'discard', '~> 1.2'
+gem 'doorkeeper', '~> 5.4'
+gem 'ed25519', '~> 1.2'
 gem 'fast_blank', '~> 1.0'
 gem 'fastimage'
-gem 'goldfinger', '~> 2.1'
 gem 'hiredis', '~> 0.6'
-gem 'redis-namespace', '~> 1.5'
+gem 'redis-namespace', '~> 1.7'
+gem 'health_check', git: 'https://github.com/ianheggie/health_check', ref: '0b799ead604f900ed50685e9b2d469cd2befba5b'
 gem 'htmlentities', '~> 4.3'
-gem 'http', '~> 3.3'
+gem 'http', '~> 4.4'
 gem 'http_accept_language', '~> 2.1'
-gem 'http_parser.rb', '~> 0.6', git: 'https://github.com/tmm1/http_parser.rb', ref: '54b17ba8c7d8d20a16dfc65d1775241833219cf2'
-gem 'httplog', '~> 1.1'
+gem 'http_parser.rb', '~> 0.6', git: 'https://github.com/tmm1/http_parser.rb', ref: '54b17ba8c7d8d20a16dfc65d1775241833219cf2', submodules: true
+gem 'httplog', '~> 1.4.3'
 gem 'idn-ruby', require: 'idn'
-gem 'kaminari', '~> 1.1'
+gem 'kaminari', '~> 1.2'
 gem 'link_header', '~> 0.0'
-gem 'mime-types', '~> 3.2', require: 'mime/types/columnar'
-gem 'nokogiri', '~> 1.8'
+gem 'mime-types', '~> 3.3.1', require: 'mime/types/columnar'
+gem 'nilsimsa', git: 'https://github.com/witgo/nilsimsa', ref: 'fd184883048b922b176939f851338d0a4971a532'
+gem 'nokogiri', '~> 1.10'
 gem 'nsa', '~> 0.2'
-gem 'oj', '~> 3.7'
-gem 'ostatus2', '~> 2.0'
-gem 'ox', '~> 2.10'
+gem 'oj', '~> 3.10'
+gem 'ox', '~> 2.13'
+gem 'parslet'
+gem 'parallel', '~> 1.19'
 gem 'posix-spawn', git: 'https://github.com/rtomayko/posix-spawn', ref: '58465d2e213991f8afb13b984854a49fcdcc980c'
-gem 'pundit', '~> 2.0'
+gem 'pundit', '~> 2.1'
 gem 'premailer-rails'
-gem 'rack-attack', '~> 5.4'
-gem 'rack-cors', '~> 1.0', require: 'rack/cors'
+gem 'rack-attack', '~> 6.3'
+gem 'rack-cors', '~> 1.1', require: 'rack/cors'
 gem 'rails-i18n', '~> 5.1'
 gem 'rails-settings-cached', '~> 0.6'
-gem 'redis', '~> 4.0', require: ['redis', 'redis/connection/hiredis']
+gem 'redis', '~> 4.2', require: ['redis', 'redis/connection/hiredis']
 gem 'mario-redis-lock', '~> 1.2', require: 'redis_lock'
-gem 'rqrcode', '~> 0.10'
-gem 'sanitize', '~> 5.0'
-gem 'sidekiq', '~> 5.2'
+gem 'rqrcode', '~> 1.1'
+gem 'ruby-progressbar', '~> 1.10'
+gem 'sanitize', '~> 5.2'
+gem 'sidekiq', '~> 6.0'
 gem 'sidekiq-scheduler', '~> 3.0'
-gem 'sidekiq-unique-jobs', '~> 5.0'
-gem 'sidekiq-bulk', '~>0.1.1'
-gem 'simple-navigation', '~> 4.0'
-gem 'simple_form', '~> 4.0'
+gem 'sidekiq-unique-jobs', '~> 6.0'
+gem 'sidekiq-bulk', '~>0.2.0'
+gem 'simple-navigation', '~> 4.1'
+gem 'simple_form', '~> 5.0'
 gem 'sprockets-rails', '~> 3.2', require: 'sprockets/railtie'
-gem 'stoplight', '~> 2.1.3'
-gem 'strong_migrations', '~> 0.3'
-gem 'tty-command', '~> 0.8', require: false
-gem 'tty-prompt', '~> 0.17', require: false
+gem 'stoplight', '~> 2.2.0'
+gem 'strong_migrations', '~> 0.6'
+gem 'tty-prompt', '~> 0.21', require: false
 gem 'twitter-text', '~> 1.14'
-gem 'tzinfo-data', '~> 1.2018'
-gem 'webpacker', '~> 3.5'
+gem 'tzinfo-data', '~> 1.2020'
+gem 'webpacker', '~> 5.1'
 gem 'webpush'
 
-gem 'json-ld', '~> 2.2'
-gem 'rdf-normalize', '~> 0.3'
+gem 'json-ld'
+gem 'json-ld-preloaded', '~> 3.1'
+gem 'rdf-normalize', '~> 0.4'
 
 group :development, :test do
-  gem 'fabrication', '~> 2.20'
-  gem 'fuubar', '~> 2.3'
+  gem 'fabrication', '~> 2.21'
+  gem 'fuubar', '~> 2.5'
   gem 'i18n-tasks', '~> 0.9', require: false
-  gem 'pry-byebug', '~> 3.6'
+  gem 'pry-byebug', '~> 3.9'
   gem 'pry-rails', '~> 0.3'
-  gem 'rspec-rails', '~> 3.8'
+  gem 'rspec-rails', '~> 4.0'
 end
 
 group :production, :test do
@@ -106,41 +118,44 @@ group :production, :test do
 end
 
 group :test do
-  gem 'capybara', '~> 3.10'
+  gem 'capybara', '~> 3.33'
   gem 'climate_control', '~> 0.2'
-  gem 'faker', '~> 1.9'
-  gem 'microformats', '~> 4.0'
+  gem 'faker', '~> 2.13'
+  gem 'microformats', '~> 4.2'
   gem 'rails-controller-testing', '~> 1.0'
-  gem 'rspec-sidekiq', '~> 3.0'
-  gem 'simplecov', '~> 0.16', require: false
-  gem 'webmock', '~> 3.4'
-  gem 'parallel_tests', '~> 2.26'
+  gem 'rspec-sidekiq', '~> 3.1'
+  gem 'simplecov', '~> 0.18', require: false
+  gem 'webmock', '~> 3.8'
+  gem 'parallel_tests', '~> 3.0'
+  gem 'rspec_junit_formatter', '~> 0.4'
 end
 
 group :development do
-  gem 'active_record_query_trace', '~> 1.5'
-  gem 'annotate', '~> 2.7'
-  gem 'better_errors', '~> 2.5'
+  gem 'active_record_query_trace', '~> 1.7'
+  gem 'annotate', '~> 3.1'
+  gem 'better_errors', '~> 2.7'
   gem 'binding_of_caller', '~> 0.7'
-  gem 'bullet', '~> 5.7'
-  gem 'letter_opener', '~> 1.4'
-  gem 'letter_opener_web', '~> 1.3'
+  gem 'bullet', '~> 6.1'
+  gem 'letter_opener', '~> 1.7'
+  gem 'letter_opener_web', '~> 1.4'
   gem 'memory_profiler'
-  gem 'rubocop', '~> 0.60', require: false
-  gem 'brakeman', '~> 4.3', require: false
-  gem 'bundler-audit', '~> 0.6', require: false
-  gem 'scss_lint', '~> 0.57', require: false
+  gem 'rubocop', '~> 0.86', require: false
+  gem 'rubocop-rails', '~> 2.6', require: false
+  gem 'brakeman', '~> 4.8', require: false
+  gem 'bundler-audit', '~> 0.7', require: false
 
-  gem 'capistrano', '~> 3.11'
-  gem 'capistrano-rails', '~> 1.4'
+  gem 'capistrano', '~> 3.14'
+  gem 'capistrano-rails', '~> 1.5'
   gem 'capistrano-rbenv', '~> 2.1'
   gem 'capistrano-yarn', '~> 2.0'
 
-  gem 'derailed_benchmarks'
   gem 'stackprof'
 end
 
 group :production do
-  gem 'lograge', '~> 0.10'
+  gem 'lograge', '~> 0.11'
   gem 'redis-rails', '~> 5.0'
 end
+
+gem 'concurrent-ruby', require: false
+gem 'connection_pool', require: false

Procfile

@@ -1,2 +1,14 @@
-web: bundle exec puma -C config/puma.rb
+web: if [ "$RUN_STREAMING" != "true" ]; then BIND=0.0.0.0 bundle exec puma -C config/puma.rb; else BIND=0.0.0.0 node ./streaming; fi
 worker: bundle exec sidekiq
+
+# For the streaming API, you need a separate app that shares Postgres and Redis:
+#
+# heroku create
+# heroku buildpacks:add heroku/nodejs
+# heroku config:set RUN_STREAMING=true
+# heroku addons:attach <main-app>::DATABASE
+# heroku addons:attach <main-app>::REDIS
+#
+# and let the main app use the separate app:
+#
+# heroku config:set STREAMING_API_BASE_URL=wss://<streaming-app>.herokuapp.com -a <main-app>

README.md

@@ -4,16 +4,16 @@
 [![GitHub release](https://img.shields.io/github/release/tootsuite/mastodon.svg)][releases]
 [![Build Status](https://img.shields.io/circleci/project/github/tootsuite/mastodon.svg)][circleci]
 [![Code Climate](https://img.shields.io/codeclimate/maintainability/tootsuite/mastodon.svg)][code_climate]
-[![Translation status](https://weblate.joinmastodon.org/widgets/mastodon/-/svg-badge.svg)][weblate]
+[![Crowdin](https://d322cqt584bo4o.cloudfront.net/mastodon/localized.svg)][crowdin]
 [![Docker Pulls](https://img.shields.io/docker/pulls/tootsuite/mastodon.svg)][docker]
 
 [releases]: https://github.com/tootsuite/mastodon/releases
 [circleci]: https://circleci.com/gh/tootsuite/mastodon
 [code_climate]: https://codeclimate.com/github/tootsuite/mastodon
-[weblate]: https://weblate.joinmastodon.org/engage/mastodon/
+[crowdin]: https://crowdin.com/project/mastodon
 [docker]: https://hub.docker.com/r/tootsuite/mastodon/
 
-Mastodon is a **free, open-source social network server** based on ActivityPub. Follow friends and discover new ones. Publish anything you want: links, pictures, text, video. All servers of Mastodon are interoperable as a federated network, i.e. users on one server can seamlessly communicate with users from another one. This includes non-Mastodon software that also implements ActivityPub!
+Mastodon is a **free, open-source social network server** based on ActivityPub where users can follow friends and discover new ones. On Mastodon, users can publish anything they want: links, pictures, text, video. All Mastodon servers are interoperable as a federated network (users on one server can seamlessly communicate with users from another one, including non-Mastodon software that implements ActivityPub)!
 
 Click below to **learn more** in a video:
 
@@ -21,7 +21,7 @@ Click below to **learn more** in a video:
 
 [youtube_demo]: https://www.youtube.com/watch?v=IPSbNdBmWKE
 
-## Navigation 
+## Navigation
 
 - [Project homepage 🐘](https://joinmastodon.org)
 - [Support the development via Patreon][patreon]
@@ -55,7 +55,7 @@ Private posts, locked accounts, phrase filtering, muting, blocking and all sorts
 
 **OAuth2 and a straightforward REST API**
 
-Mastodon acts as an OAuth2 provider so 3rd party apps can use the REST and Streaming APIs, resulting in a rich app ecosystem with a lot of choice!
+Mastodon acts as an OAuth2 provider so 3rd party apps can use the REST and Streaming APIs, resulting in a rich app ecosystem with a lot of choices!
 
 ## Deployment
 
@@ -68,25 +68,25 @@ Mastodon acts as an OAuth2 provider so 3rd party apps can use the REST and Strea
 **Requirements:**
 
 - **PostgreSQL** 9.5+
-- **Redis**
-- **Ruby** 2.4+
-- **Node.js** 8+
+- **Redis** 4+
+- **Ruby** 2.5+
+- **Node.js** 10.13+
 
-The repository includes deployment configurations for **Docker and docker-compose**, but also a few specific platforms like **Heroku**, **Scalingo**, and **Nanobox**. The [**stand-alone** installation guide](https://docs.joinmastodon.org/administration/installation/) is available in the documentation.
+The repository includes deployment configurations for **Docker and docker-compose**, but also a few specific platforms like **Heroku**, **Scalingo**, and **Nanobox**. The [**stand-alone** installation guide](https://docs.joinmastodon.org/admin/install/) is available in the documentation.
 
 A **Vagrant** configuration is included for development purposes.
 
 ## Contributing
 
-Mastodon is **free, open source software** licensed under **AGPLv3**.
+Mastodon is **free, open-source software** licensed under **AGPLv3**.
 
-You can open issues for bugs you've found or features you think are missing. You can also submit pull requests to this repository, or submit translations using Weblate. To get started, take a look at [CONTRIBUTING.md](CONTRIBUTING.md)
+You can open issues for bugs you've found or features you think are missing. You can also submit pull requests to this repository, or submit translations using Crowdin. To get started, take a look at [CONTRIBUTING.md](CONTRIBUTING.md). If your contributions are accepted into Mastodon, you can request to be paid through [our OpenCollective](https://opencollective.com/mastodon).
 
 **IRC channel**: #mastodon on irc.freenode.net
 
 ## License
 
-Copyright (C) 2016-2018 Eugen Rochko & other Mastodon contributors (see [AUTHORS.md](AUTHORS.md))
+Copyright (C) 2016-2020 Eugen Rochko & other Mastodon contributors (see [AUTHORS.md](AUTHORS.md))
 
 This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
 

SECURITY.md

@@ -0,0 +1,12 @@
+# Security Policy
+
+## Supported Versions
+
+| Version | Supported          |
+| ------- | ------------------ |
+| 3.1.x   | :white_check_mark: |
+| < 3.1   | :x:                |
+
+## Reporting a Vulnerability
+
+hello@joinmastodon.org

Vagrantfile

@@ -12,7 +12,7 @@ curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -
 sudo apt-add-repository 'deb https://dl.yarnpkg.com/debian/ stable main'
 
 # Add repo for NodeJS
-curl -sL https://deb.nodesource.com/setup_8.x | sudo bash -
+curl -sL https://deb.nodesource.com/setup_10.x | sudo bash -
 
 # Add firewall rule to redirect 80 to PORT and save
 sudo iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port #{ENV["PORT"]}
@@ -44,7 +44,18 @@ sudo apt-get install \
 
 # Install rvm
 read RUBY_VERSION < .ruby-version
-gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
+
+gpg_command="gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB"
+$($gpg_command)
+if [ $? -ne 0 ];then
+  echo "GPG command failed, This prevented RVM from installing."
+  echo "Retrying once..." && $($gpg_command)
+  if [ $? -ne 0 ];then
+    echo "GPG failed for the second time, please ensure network connectivity."
+    echo "Exiting..." && exit 1
+  fi
+fi
+
 curl -sSL https://raw.githubusercontent.com/rvm/rvm/stable/binscripts/rvm-installer | bash -s stable --ruby=$RUBY_VERSION
 source /home/vagrant/.rvm/scripts/rvm
 
@@ -80,7 +91,7 @@ VAGRANTFILE_API_VERSION = "2"
 
 Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
 
-  config.vm.box = "ubuntu/xenial64"
+  config.vm.box = "ubuntu/bionic64"
 
   config.vm.provider :virtualbox do |vb|
     vb.name = "mastodon"

app.json

@@ -13,15 +13,6 @@
       "description": "The domain that your Mastodon instance will run on (this can be appname.herokuapp.com or a custom domain)",
       "required": true
     },
-    "LOCAL_HTTPS": {
-      "description": "Will your domain support HTTPS? (Automatic for herokuapp, requires manual configuration for custom domains)",
-      "value": "false",
-      "required": true
-    },
-    "PAPERCLIP_SECRET": {
-      "description": "The secret key for storing media files",
-      "generator": "secret"
-    },
     "SECRET_KEY_BASE": {
       "description": "The secret key base",
       "generator": "secret"
@@ -97,9 +88,6 @@
     {
       "url": "https://github.com/heroku/heroku-buildpack-apt"
     },
-    {
-      "url": "heroku/nodejs"
-    },
     {
       "url": "heroku/ruby"
     }

app/chewy/accounts_index.rb

@@ -0,0 +1,43 @@
+# frozen_string_literal: true
+
+class AccountsIndex < Chewy::Index
+  settings index: { refresh_interval: '5m' }, analysis: {
+    analyzer: {
+      content: {
+        tokenizer: 'whitespace',
+        filter: %w(lowercase asciifolding cjk_width),
+      },
+
+      edge_ngram: {
+        tokenizer: 'edge_ngram',
+        filter: %w(lowercase asciifolding cjk_width),
+      },
+    },
+
+    tokenizer: {
+      edge_ngram: {
+        type: 'edge_ngram',
+        min_gram: 1,
+        max_gram: 15,
+      },
+    },
+  }
+
+  define_type ::Account.searchable.includes(:account_stat), delete_if: ->(account) { account.destroyed? || !account.searchable? } do
+    root date_detection: false do
+      field :id, type: 'long'
+
+      field :display_name, type: 'text', analyzer: 'content' do
+        field :edge_ngram, type: 'text', analyzer: 'edge_ngram', search_analyzer: 'content'
+      end
+
+      field :acct, type: 'text', analyzer: 'content', value: ->(account) { [account.username, account.domain].compact.join('@') } do
+        field :edge_ngram, type: 'text', analyzer: 'edge_ngram', search_analyzer: 'content'
+      end
+
+      field :following_count, type: 'long', value: ->(account) { account.following.local.count }
+      field :followers_count, type: 'long', value: ->(account) { account.followers.local.count }
+      field :last_status_at, type: 'date', value: ->(account) { account.last_status_at || account.created_at }
+    end
+  end
+end

app/chewy/statuses_index.rb

@@ -31,31 +31,36 @@ class StatusesIndex < Chewy::Index
     },
   }
 
-  define_type ::Status.unscoped.without_reblogs do
+  define_type ::Status.unscoped.kept.without_reblogs.includes(:media_attachments, :preloadable_poll) do
     crutch :mentions do |collection|
-      data = ::Mention.where(status_id: collection.map(&:id)).pluck(:status_id, :account_id)
+      data = ::Mention.where(status_id: collection.map(&:id)).where(account: Account.local, silent: false).pluck(:status_id, :account_id)
       data.each.with_object({}) { |(id, name), result| (result[id] ||= []).push(name) }
     end
 
     crutch :favourites do |collection|
-      data = ::Favourite.where(status_id: collection.map(&:id)).pluck(:status_id, :account_id)
+      data = ::Favourite.where(status_id: collection.map(&:id)).where(account: Account.local).pluck(:status_id, :account_id)
       data.each.with_object({}) { |(id, name), result| (result[id] ||= []).push(name) }
     end
 
     crutch :reblogs do |collection|
-      data = ::Status.where(reblog_of_id: collection.map(&:id)).pluck(:reblog_of_id, :account_id)
+      data = ::Status.where(reblog_of_id: collection.map(&:id)).where(account: Account.local).pluck(:reblog_of_id, :account_id)
+      data.each.with_object({}) { |(id, name), result| (result[id] ||= []).push(name) }
+    end
+
+    crutch :bookmarks do |collection|
+      data = ::Bookmark.where(status_id: collection.map(&:id)).where(account: Account.local).pluck(:status_id, :account_id)
       data.each.with_object({}) { |(id, name), result| (result[id] ||= []).push(name) }
     end
 
     root date_detection: false do
+      field :id, type: 'long'
       field :account_id, type: 'long'
 
-      field :text, type: 'text', value: ->(status) { [status.spoiler_text, Formatter.instance.plaintext(status)].join("\n\n") } do
+      field :text, type: 'text', value: ->(status) { [status.spoiler_text, Formatter.instance.plaintext(status)].concat(status.media_attachments.map(&:description)).concat(status.preloadable_poll ? status.preloadable_poll.options : []).join("\n\n") } do
         field :stemmed, type: 'text', analyzer: 'content'
       end
 
       field :searchable_by, type: 'long', value: ->(status, crutches) { status.searchable_by(crutches) }
-      field :created_at, type: 'date'
     end
   end
 end

app/chewy/tags_index.rb

@@ -0,0 +1,37 @@
+# frozen_string_literal: true
+
+class TagsIndex < Chewy::Index
+  settings index: { refresh_interval: '15m' }, analysis: {
+    analyzer: {
+      content: {
+        tokenizer: 'keyword',
+        filter: %w(lowercase asciifolding cjk_width),
+      },
+
+      edge_ngram: {
+        tokenizer: 'edge_ngram',
+        filter: %w(lowercase asciifolding cjk_width),
+      },
+    },
+
+    tokenizer: {
+      edge_ngram: {
+        type: 'edge_ngram',
+        min_gram: 2,
+        max_gram: 15,
+      },
+    },
+  }
+
+  define_type ::Tag.listable, delete_if: ->(tag) { tag.destroyed? || !tag.listable? } do
+    root date_detection: false do
+      field :name, type: 'text', analyzer: 'content' do
+        field :edge_ngram, type: 'text', analyzer: 'edge_ngram', search_analyzer: 'content'
+      end
+
+      field :reviewed, type: 'boolean', value: ->(tag) { tag.reviewed? }
+      field :usage, type: 'long', value: ->(tag) { tag.history.reduce(0) { |total, day| total + day[:accounts].to_i } }
+      field :last_status_at, type: 'date', value: ->(tag) { tag.last_status_at || tag.created_at }
+    end
+  end
+end

app/controllers/about_controller.rb

@@ -1,42 +1,64 @@
 # frozen_string_literal: true
 
 class AboutController < ApplicationController
-  before_action :set_body_classes
-  before_action :set_instance_presenter, only: [:show, :more, :terms]
+  layout 'public'
 
-  def show
-    serializable_resource = ActiveModelSerializers::SerializableResource.new(InitialStatePresenter.new(initial_state_params), serializer: InitialStateSerializer)
-    @initial_state_json   = serializable_resource.to_json
-  end
+  before_action :require_open_federation!, only: [:show, :more]
+  before_action :set_body_classes, only: :show
+  before_action :set_instance_presenter
+  before_action :set_expires_in, only: [:show, :more, :terms]
+
+  skip_before_action :require_functional!, only: [:more, :terms]
+
+  def show; end
 
   def more
-    render layout: 'public'
+    flash.now[:notice] = I18n.t('about.instance_actor_flash') if params[:instance_actor]
+
+    toc_generator = TOCGenerator.new(@instance_presenter.site_extended_description)
+
+    @contents          = toc_generator.html
+    @table_of_contents = toc_generator.toc
+    @blocks            = DomainBlock.with_user_facing_limitations.by_severity if display_blocks?
   end
 
-  def terms
-    render layout: 'public'
-  end
+  def terms; end
+
+  helper_method :display_blocks?
+  helper_method :display_blocks_rationale?
+  helper_method :public_fetch_mode?
+  helper_method :new_user
 
   private
 
-  def new_user
-    User.new.tap(&:build_account)
+  def require_open_federation!
+    not_found if whitelist_mode?
   end
 
-  helper_method :new_user
+  def display_blocks?
+    Setting.show_domain_blocks == 'all' || (Setting.show_domain_blocks == 'users' && user_signed_in?)
+  end
+
+  def display_blocks_rationale?
+    Setting.show_domain_blocks_rationale == 'all' || (Setting.show_domain_blocks_rationale == 'users' && user_signed_in?)
+  end
+
+  def new_user
+    User.new.tap do |user|
+      user.build_account
+      user.build_invite_request
+    end
+  end
 
   def set_instance_presenter
     @instance_presenter = InstancePresenter.new
   end
 
   def set_body_classes
-    @body_classes = 'with-modals'
+    @hide_navbar = true
   end
 
-  def initial_state_params
-    {
-      settings: { known_fediverse: Setting.show_known_fediverse_at_about_page },
-      token: current_session&.token,
-    }
+  def set_expires_in
+    expires_in 0, public: true
   end
 end

app/controllers/account_follow_controller.rb

@@ -6,7 +6,7 @@ class AccountFollowController < ApplicationController
   before_action :authenticate_user!
 
   def create
-    FollowService.new.call(current_user.account, @account.acct)
+    FollowService.new.call(current_user.account, @account, with_rate_limit: true)
     redirect_to account_path(@account)
   end
 end

app/controllers/accounts_controller.rb

@@ -1,18 +1,27 @@
 # frozen_string_literal: true
 
 class AccountsController < ApplicationController
-  PAGE_SIZE = 20
+  PAGE_SIZE     = 20
+  PAGE_SIZE_MAX = 200
 
   include AccountControllerConcern
+  include SignatureAuthentication
 
+  before_action :require_signature!, if: -> { request.format == :json && authorized_fetch_mode? }
   before_action :set_cache_headers
+  before_action :set_body_classes
+
+  skip_around_action :set_locale, if: -> { [:json, :rss].include?(request.format&.to_sym) }
+  skip_before_action :require_functional!, unless: :whitelist_mode?
 
   def show
     respond_to do |format|
       format.html do
-        @body_classes      = 'with-modals'
+        expires_in 0, public: true unless user_signed_in?
+
         @pinned_statuses   = []
         @endorsed_accounts = @account.endorsed_accounts.to_a.sample(4)
+        @featured_hashtags = @account.featured_tags.order(statuses_count: :desc)
 
         if current_account && @account.blocking?(current_account)
           @statuses = []
@@ -20,8 +29,9 @@ class AccountsController < ApplicationController
         end
 
         @pinned_statuses = cache_collection(@account.pinned_statuses, Status) if show_pinned_statuses?
-        @statuses        = filtered_status_page(params)
+        @statuses        = filtered_status_page
         @statuses        = cache_collection(@statuses, Status)
+        @rss_url         = rss_url
 
         unless @statuses.empty?
           @older_url = older_url if @statuses.last.id > filtered_statuses.last.id
@@ -29,34 +39,35 @@ class AccountsController < ApplicationController
         end
       end
 
-      format.atom do
-        @entries = @account.stream_entries.where(hidden: false).with_includes.paginate_by_max_id(PAGE_SIZE, params[:max_id], params[:since_id])
-        render xml: OStatus::AtomSerializer.render(OStatus::AtomSerializer.new.feed(@account, @entries.reject { |entry| entry.status.nil? }))
-      end
-
       format.rss do
-        @statuses = cache_collection(default_statuses.without_reblogs.without_replies.limit(PAGE_SIZE), Status)
-        render xml: RSS::AccountSerializer.render(@account, @statuses)
+        expires_in 1.minute, public: true
+
+        limit     = params[:limit].present? ? [params[:limit].to_i, PAGE_SIZE_MAX].min : PAGE_SIZE
+        @statuses = filtered_statuses.without_reblogs.limit(limit)
+        @statuses = cache_collection(@statuses, Status)
+        render xml: RSS::AccountSerializer.render(@account, @statuses, params[:tag])
       end
 
       format.json do
-        skip_session!
-
-        render_cached_json(['activitypub', 'actor', @account], content_type: 'application/activity+json') do
-          ActiveModelSerializers::SerializableResource.new(@account, serializer: ActivityPub::ActorSerializer, adapter: ActivityPub::Adapter)
-        end
+        expires_in 3.minutes, public: !(authorized_fetch_mode? && signed_request_account.present?)
+        render_with_cache json: @account, content_type: 'application/activity+json', serializer: ActivityPub::ActorSerializer, adapter: ActivityPub::Adapter
       end
     end
   end
 
   private
 
+  def set_body_classes
+    @body_classes = 'with-modals'
+  end
+
   def show_pinned_statuses?
-    [replies_requested?, media_requested?, params[:max_id].present?, params[:min_id].present?].none?
+    [replies_requested?, media_requested?, tag_requested?, params[:max_id].present?, params[:min_id].present?].none?
   end
 
   def filtered_statuses
     default_statuses.tap do |statuses|
+      statuses.merge!(hashtag_scope)    if tag_requested?
       statuses.merge!(only_media_scope) if media_requested?
       statuses.merge!(no_replies_scope) unless replies_requested?
     end
@@ -78,12 +89,29 @@ class AccountsController < ApplicationController
     Status.without_replies
   end
 
-  def set_account
-    @account = Account.find_local!(params[:username])
+  def hashtag_scope
+    tag = Tag.find_normalized(params[:tag])
+
+    if tag
+      Status.tagged_with(tag.id)
+    else
+      Status.none
+    end
+  end
+
+  def username_param
+    params[:username]
+  end
+
+  def rss_url
+    if tag_requested?
+      short_account_tag_url(@account, params[:tag], format: 'rss')
+    else
+      short_account_url(@account, format: 'rss')
+    end
   end
 
   def older_url
-    ::Rails.logger.info("older: max_id #{@statuses.last.id}, url #{pagination_url(max_id: @statuses.last.id)}")
     pagination_url(max_id: @statuses.last.id)
   end
 
@@ -92,7 +120,9 @@ class AccountsController < ApplicationController
   end
 
   def pagination_url(max_id: nil, min_id: nil)
-    if media_requested?
+    if tag_requested?
+      short_account_tag_url(@account, params[:tag], max_id: max_id, min_id: min_id)
+    elsif media_requested?
       short_account_media_url(@account, max_id: max_id, min_id: min_id)
     elsif replies_requested?
       short_account_with_replies_url(@account, max_id: max_id, min_id: min_id)
@@ -102,18 +132,22 @@ class AccountsController < ApplicationController
   end
 
   def media_requested?
-    request.path.ends_with?('/media')
+    request.path.split('.').first.ends_with?('/media') && !tag_requested?
   end
 
   def replies_requested?
-    request.path.ends_with?('/with_replies')
+    request.path.split('.').first.ends_with?('/with_replies') && !tag_requested?
   end
 
-  def filtered_status_page(params)
-    if params[:min_id].present?
-      filtered_statuses.paginate_by_min_id(PAGE_SIZE, params[:min_id]).reverse
-    else
-      filtered_statuses.paginate_by_max_id(PAGE_SIZE, params[:max_id], params[:since_id]).to_a
-    end
+  def tag_requested?
+    request.path.split('.').first.ends_with?(Addressable::URI.parse("/tagged/#{params[:tag]}").normalize)
+  end
+
+  def filtered_status_page
+    filtered_statuses.paginate_by_id(PAGE_SIZE, params_slice(:max_id, :min_id, :since_id))
+  end
+
+  def params_slice(*keys)
+    params.slice(*keys).permit(*keys)
   end
 end

app/controllers/activitypub/base_controller.rb

@@ -0,0 +1,11 @@
+# frozen_string_literal: true
+
+class ActivityPub::BaseController < Api::BaseController
+  skip_before_action :require_authenticated_user!
+
+  private
+
+  def set_cache_headers
+    response.headers['Vary'] = 'Signature' if authorized_fetch_mode?
+  end
+end

app/controllers/activitypub/claims_controller.rb

@@ -0,0 +1,21 @@
+# frozen_string_literal: true
+
+class ActivityPub::ClaimsController < ActivityPub::BaseController
+  include SignatureVerification
+  include AccountOwnedConcern
+
+  skip_before_action :authenticate_user!
+
+  before_action :require_signature!
+  before_action :set_claim_result
+
+  def create
+    render json: @claim_result, serializer: ActivityPub::OneTimeKeySerializer
+  end
+
+  private
+
+  def set_claim_result
+    @claim_result = ::Keys::ClaimService.new.call(@account.id, params[:id])
+  end
+end

app/controllers/activitypub/collections_controller.rb

@@ -1,57 +1,69 @@
 # frozen_string_literal: true
 
-class ActivityPub::CollectionsController < Api::BaseController
+class ActivityPub::CollectionsController < ActivityPub::BaseController
   include SignatureVerification
+  include AccountOwnedConcern
 
-  before_action :set_account
+  before_action :require_signature!, if: :authorized_fetch_mode?
+  before_action :set_items
   before_action :set_size
-  before_action :set_statuses
+  before_action :set_type
+  before_action :set_cache_headers
 
   def show
-    render json: collection_presenter,
-           serializer: ActivityPub::CollectionSerializer,
-           adapter: ActivityPub::Adapter,
-           content_type: 'application/activity+json',
-           skip_activities: true
+    expires_in 3.minutes, public: public_fetch_mode?
+    render_with_cache json: collection_presenter, content_type: 'application/activity+json', serializer: ActivityPub::CollectionSerializer, adapter: ActivityPub::Adapter, skip_activities: true
   end
 
   private
 
-  def set_account
-    @account = Account.find_local!(params[:account_username])
-  end
+  def set_items
+    case params[:id]
+    when 'featured'
+      @items = begin
+        # Because in public fetch mode we cache the response, there would be no
+        # benefit from performing the check below, since a blocked account or domain
+        # would likely be served the cache from the reverse proxy anyway
 
-  def set_statuses
-    @statuses = scope_for_collection
-    @statuses = cache_collection(@statuses, Status)
+        if authorized_fetch_mode? && !signed_request_account.nil? && (@account.blocking?(signed_request_account) || (!signed_request_account.domain.nil? && @account.domain_blocking?(signed_request_account.domain)))
+          []
+        else
+          cache_collection(@account.pinned_statuses, Status)
+        end
+      end
+    when 'devices'
+      @items = @account.devices
+    else
+      not_found
+    end
   end
 
   def set_size
     case params[:id]
-    when 'featured'
-      @account.pinned_statuses.count
+    when 'featured', 'devices'
+      @size = @items.size
     else
-      raise ActiveRecord::NotFound
+      not_found
     end
   end
 
-  def scope_for_collection
+  def set_type
     case params[:id]
     when 'featured'
-      @account.statuses.permitted_for(@account, signed_request_account).tap do |scope|
-        scope.merge!(@account.pinned_statuses)
-      end
+      @type = :ordered
+    when 'devices'
+      @type = :unordered
     else
-      raise ActiveRecord::NotFound
+      not_found
     end
   end
 
   def collection_presenter
     ActivityPub::CollectionPresenter.new(
       id: account_collection_url(@account, params[:id]),
-      type: :ordered,
+      type: @type,
       size: @size,
-      items: @statuses
+      items: @items
     )
   end
 end

app/controllers/activitypub/inboxes_controller.rb

@@ -1,28 +1,46 @@
 # frozen_string_literal: true
 
-class ActivityPub::InboxesController < Api::BaseController
+class ActivityPub::InboxesController < ActivityPub::BaseController
   include SignatureVerification
+  include JsonLdHelper
+  include AccountOwnedConcern
 
-  before_action :set_account
+  before_action :skip_unknown_actor_delete
+  before_action :require_signature!
+  skip_before_action :authenticate_user!
 
   def create
-    if signed_request_account
-      upgrade_account
-      process_payload
-      head 202
-    else
-      render plain: signature_verification_failure_reason, status: 401
-    end
+    upgrade_account
+    process_payload
+    head 202
   end
 
   private
 
-  def set_account
-    @account = Account.find_local!(params[:account_username]) if params[:account_username]
+  def skip_unknown_actor_delete
+    head 202 if unknown_deleted_account?
+  end
+
+  def unknown_deleted_account?
+    json = Oj.load(body, mode: :strict)
+    json.is_a?(Hash) && json['type'] == 'Delete' && json['actor'].present? && json['actor'] == value_or_id(json['object']) && !Account.where(uri: json['actor']).exists?
+  rescue Oj::ParseError
+    false
+  end
+
+  def account_required?
+    params[:account_username].present?
   end
 
   def body
-    @body ||= request.body.read
+    return @body if defined?(@body)
+
+    @body = request.body.read
+    @body.force_encoding('UTF-8') if @body.present?
+
+    request.body.rewind if request.body.respond_to?(:rewind)
+
+    @body
   end
 
   def upgrade_account
@@ -31,11 +49,10 @@ class ActivityPub::InboxesController < Api::BaseController
       ResolveAccountWorker.perform_async(signed_request_account.acct)
     end
 
-    Pubsubhubbub::UnsubscribeWorker.perform_async(signed_request_account.id) if signed_request_account.subscribed?
-    DeliveryFailureTracker.track_inverse_success!(signed_request_account)
+    DeliveryFailureTracker.reset!(signed_request_account.inbox_url)
   end
 
   def process_payload
-    ActivityPub::ProcessingWorker.perform_async(signed_request_account.id, body.force_encoding('UTF-8'), @account&.id)
+    ActivityPub::ProcessingWorker.perform_async(signed_request_account.id, body, @account&.id)
   end
 end

app/controllers/activitypub/outboxes_controller.rb

@@ -1,23 +1,22 @@
 # frozen_string_literal: true
 
-class ActivityPub::OutboxesController < Api::BaseController
+class ActivityPub::OutboxesController < ActivityPub::BaseController
   LIMIT = 20
 
   include SignatureVerification
+  include AccountOwnedConcern
 
-  before_action :set_account
+  before_action :require_signature!, if: :authorized_fetch_mode?
   before_action :set_statuses
+  before_action :set_cache_headers
 
   def show
+    expires_in(page_requested? ? 0 : 3.minutes, public: public_fetch_mode? && !(signed_request_account.present? && page_requested?))
     render json: outbox_presenter, serializer: ActivityPub::OutboxSerializer, adapter: ActivityPub::Adapter, content_type: 'application/activity+json'
   end
 
   private
 
-  def set_account
-    @account = Account.find_local!(params[:account_username])
-  end
-
   def outbox_presenter
     if page_requested?
       ActivityPub::CollectionPresenter.new(
@@ -51,12 +50,12 @@ class ActivityPub::OutboxesController < Api::BaseController
     return unless page_requested?
 
     @statuses = @account.statuses.permitted_for(@account, signed_request_account)
-    @statuses = params[:min_id].present? ? @statuses.paginate_by_min_id(LIMIT, params[:min_id]).reverse : @statuses.paginate_by_max_id(LIMIT, params[:max_id])
+    @statuses = @statuses.paginate_by_id(LIMIT, params_slice(:max_id, :min_id, :since_id))
     @statuses = cache_collection(@statuses, Status)
   end
 
   def page_requested?
-    params[:page] == 'true'
+    truthy_param?(:page)
   end
 
   def page_params

app/controllers/activitypub/replies_controller.rb

@@ -0,0 +1,80 @@
+# frozen_string_literal: true
+
+class ActivityPub::RepliesController < ActivityPub::BaseController
+  include SignatureVerification
+  include Authorization
+  include AccountOwnedConcern
+
+  DESCENDANTS_LIMIT = 60
+
+  before_action :require_signature!, if: :authorized_fetch_mode?
+  before_action :set_status
+  before_action :set_cache_headers
+  before_action :set_replies
+
+  def index
+    expires_in 0, public: public_fetch_mode?
+    render json: replies_collection_presenter, serializer: ActivityPub::CollectionSerializer, adapter: ActivityPub::Adapter, content_type: 'application/activity+json', skip_activities: true
+  end
+
+  private
+
+  def pundit_user
+    signed_request_account
+  end
+
+  def set_status
+    @status = @account.statuses.find(params[:status_id])
+    authorize @status, :show?
+  rescue Mastodon::NotPermittedError
+    not_found
+  end
+
+  def set_replies
+    @replies = only_other_accounts? ? Status.where.not(account_id: @account.id) : @account.statuses
+    @replies = @replies.where(in_reply_to_id: @status.id, visibility: [:public, :unlisted])
+    @replies = @replies.paginate_by_min_id(DESCENDANTS_LIMIT, params[:min_id])
+  end
+
+  def replies_collection_presenter
+    page = ActivityPub::CollectionPresenter.new(
+      id: account_status_replies_url(@account, @status, page_params),
+      type: :unordered,
+      part_of: account_status_replies_url(@account, @status),
+      next: next_page,
+      items: @replies.map { |status| status.local? ? status : status.uri }
+    )
+
+    return page if page_requested?
+
+    ActivityPub::CollectionPresenter.new(
+      id: account_status_replies_url(@account, @status),
+      type: :unordered,
+      first: page
+    )
+  end
+
+  def page_requested?
+    truthy_param?(:page)
+  end
+
+  def only_other_accounts?
+    truthy_param?(:only_other_accounts)
+  end
+
+  def next_page
+    only_other_accounts = !(@replies&.last&.account_id == @account.id && @replies.size == DESCENDANTS_LIMIT)
+
+    account_status_replies_url(
+      @account,
+      @status,
+      page: true,
+      min_id: only_other_accounts && !only_other_accounts? ? nil : @replies&.last&.id,
+      only_other_accounts: only_other_accounts
+    )
+  end
+
+  def page_params
+    params_slice(:only_other_accounts, :min_id).merge(page: true)
+  end
+end

app/controllers/admin/account_actions_controller.rb

@@ -0,0 +1,36 @@
+# frozen_string_literal: true
+
+module Admin
+  class AccountActionsController < BaseController
+    before_action :set_account
+
+    def new
+      @account_action  = Admin::AccountAction.new(type: params[:type], report_id: params[:report_id], send_email_notification: true, include_statuses: true)
+      @warning_presets = AccountWarningPreset.all
+    end
+
+    def create
+      account_action                 = Admin::AccountAction.new(resource_params)
+      account_action.target_account  = @account
+      account_action.current_account = current_account
+
+      account_action.save!
+
+      if account_action.with_report?
+        redirect_to admin_reports_path
+      else
+        redirect_to admin_account_path(@account.id)
+      end
+    end
+
+    private
+
+    def set_account
+      @account = Account.find(params[:account_id])
+    end
+
+    def resource_params
+      params.require(:admin_account_action).permit(:type, :report_id, :warning_preset_id, :text, :send_email_notification, :include_statuses)
+    end
+  end
+end

app/controllers/admin/account_moderation_notes_controller.rb

@@ -14,6 +14,7 @@ module Admin
       else
         @account          = @account_moderation_note.target_account
         @moderation_notes = @account.targeted_moderation_notes.latest
+        @warnings         = @account.targeted_account_warnings.latest.custom
 
         render template: 'admin/accounts/show'
       end

app/controllers/admin/accounts_controller.rb

@@ -2,9 +2,9 @@
 
 module Admin
   class AccountsController < BaseController
-    before_action :set_account, only: [:show, :subscribe, :unsubscribe, :redownload, :remove_avatar, :enable, :disable, :memorialize]
-    before_action :require_remote_account!, only: [:subscribe, :unsubscribe, :redownload]
-    before_action :require_local_account!, only: [:enable, :disable, :memorialize]
+    before_action :set_account, only: [:show, :redownload, :remove_avatar, :remove_header, :enable, :unsilence, :unsuspend, :memorialize, :approve, :reject]
+    before_action :require_remote_account!, only: [:redownload]
+    before_action :require_local_account!, only: [:enable, :memorialize, :approve, :reject]
 
     def index
       authorize :account, :index?
@@ -13,20 +13,10 @@ module Admin
 
     def show
       authorize @account, :show?
+
       @account_moderation_note = current_account.account_moderation_notes.new(target_account: @account)
-      @moderation_notes = @account.targeted_moderation_notes.latest
-    end
-
-    def subscribe
-      authorize @account, :subscribe?
-      Pubsubhubbub::SubscribeWorker.perform_async(@account.id)
-      redirect_to admin_account_path(@account.id)
-    end
-
-    def unsubscribe
-      authorize @account, :unsubscribe?
-      Pubsubhubbub::UnsubscribeWorker.perform_async(@account.id)
-      redirect_to admin_account_path(@account.id)
+      @moderation_notes        = @account.targeted_moderation_notes.latest
+      @warnings                = @account.targeted_account_warnings.latest.custom
     end
 
     def memorialize
@@ -43,19 +33,37 @@ module Admin
       redirect_to admin_account_path(@account.id)
     end
 
-    def disable
-      authorize @account.user, :disable?
-      @account.user.disable!
-      log_action :disable, @account.user
+    def approve
+      authorize @account.user, :approve?
+      @account.user.approve!
+      redirect_to admin_pending_accounts_path
+    end
+
+    def reject
+      authorize @account.user, :reject?
+      SuspendAccountService.new.call(@account, reserve_email: false, reserve_username: false)
+      redirect_to admin_pending_accounts_path
+    end
+
+    def unsilence
+      authorize @account, :unsilence?
+      @account.unsilence!
+      log_action :unsilence, @account
+      redirect_to admin_account_path(@account.id)
+    end
+
+    def unsuspend
+      authorize @account, :unsuspend?
+      @account.unsuspend!
+      log_action :unsuspend, @account
       redirect_to admin_account_path(@account.id)
     end
 
     def redownload
       authorize @account, :redownload?
 
-      @account.reset_avatar!
-      @account.reset_header!
-      @account.save!
+      @account.update!(last_webfingered_at: nil)
+      ResolveAccountService.new.call(@account)
 
       redirect_to admin_account_path(@account.id)
     end
@@ -71,6 +79,17 @@ module Admin
       redirect_to admin_account_path(@account.id)
     end
 
+    def remove_header
+      authorize @account, :remove_header?
+
+      @account.header = nil
+      @account.save!
+
+      log_action :remove_header, @account.user
+
+      redirect_to admin_account_path(@account.id)
+    end
+
     private
 
     def set_account
@@ -90,19 +109,7 @@ module Admin
     end
 
     def filter_params
-      params.permit(
-        :local,
-        :remote,
-        :by_domain,
-        :silenced,
-        :alphabetic,
-        :suspended,
-        :username,
-        :display_name,
-        :email,
-        :ip,
-        :staff
-      )
+      params.slice(*AccountFilter::KEYS).permit(*AccountFilter::KEYS)
     end
   end
 end

app/controllers/admin/action_logs_controller.rb

@@ -2,8 +2,18 @@
 
 module Admin
   class ActionLogsController < BaseController
-    def index
-      @action_logs = Admin::ActionLog.page(params[:page])
+    before_action :set_action_logs
+
+    def index; end
+
+    private
+
+    def set_action_logs
+      @action_logs = Admin::ActionLogFilter.new(filter_params).results.page(params[:page])
+    end
+
+    def filter_params
+      params.slice(:page, *Admin::ActionLogFilter::KEYS).permit(:page, *Admin::ActionLogFilter::KEYS)
     end
   end
 end

app/controllers/admin/announcements_controller.rb

@@ -0,0 +1,88 @@
+# frozen_string_literal: true
+
+class Admin::AnnouncementsController < Admin::BaseController
+  before_action :set_announcements, only: :index
+  before_action :set_announcement, except: [:index, :new, :create]
+
+  def index
+    authorize :announcement, :index?
+  end
+
+  def new
+    authorize :announcement, :create?
+
+    @announcement = Announcement.new
+  end
+
+  def create
+    authorize :announcement, :create?
+
+    @announcement = Announcement.new(resource_params)
+
+    if @announcement.save
+      PublishScheduledAnnouncementWorker.perform_async(@announcement.id) if @announcement.published?
+      log_action :create, @announcement
+      redirect_to admin_announcements_path, notice: @announcement.published? ? I18n.t('admin.announcements.published_msg') : I18n.t('admin.announcements.scheduled_msg')
+    else
+      render :new
+    end
+  end
+
+  def edit
+    authorize :announcement, :update?
+  end
+
+  def update
+    authorize :announcement, :update?
+
+    if @announcement.update(resource_params)
+      PublishScheduledAnnouncementWorker.perform_async(@announcement.id) if @announcement.published?
+      log_action :update, @announcement
+      redirect_to admin_announcements_path, notice: I18n.t('admin.announcements.updated_msg')
+    else
+      render :edit
+    end
+  end
+
+  def publish
+    authorize :announcement, :update?
+    @announcement.publish!
+    PublishScheduledAnnouncementWorker.perform_async(@announcement.id)
+    log_action :update, @announcement
+    redirect_to admin_announcements_path, notice: I18n.t('admin.announcements.published_msg')
+  end
+
+  def unpublish
+    authorize :announcement, :update?
+    @announcement.unpublish!
+    UnpublishAnnouncementWorker.perform_async(@announcement.id)
+    log_action :update, @announcement
+    redirect_to admin_announcements_path, notice: I18n.t('admin.announcements.unpublished_msg')
+  end
+
+  def destroy
+    authorize :announcement, :destroy?
+    @announcement.destroy!
+    UnpublishAnnouncementWorker.perform_async(@announcement.id) if @announcement.published?
+    log_action :destroy, @announcement
+    redirect_to admin_announcements_path, notice: I18n.t('admin.announcements.destroyed_msg')
+  end
+
+  private
+
+  def set_announcements
+    @announcements = AnnouncementFilter.new(filter_params).results.page(params[:page])
+  end
+
+  def set_announcement
+    @announcement = Announcement.find(params[:id])
+  end
+
+  def filter_params
+    params.slice(*AnnouncementFilter::KEYS).permit(*AnnouncementFilter::KEYS)
+  end
+
+  def resource_params
+    params.require(:announcement).permit(:text, :scheduled_at, :starts_at, :ends_at, :all_day)
+  end
+end

app/controllers/admin/base_controller.rb

@@ -15,5 +15,9 @@ module Admin
     def set_body_classes
       @body_classes = 'admin'
     end
+
+    def set_user
+      @user = Account.find(params[:account_id]).user || raise(ActiveRecord::RecordNotFound)
+    end
   end
 end

app/controllers/admin/confirmations_controller.rb

@@ -25,10 +25,6 @@ module Admin
 
     private
 
-    def set_user
-      @user = Account.find(params[:account_id]).user || raise(ActiveRecord::RecordNotFound)
-    end
-
     def check_confirmation
       if @user.confirmed?
         flash[:error] = I18n.t('admin.accounts.resend_confirmation.already_confirmed')

app/controllers/admin/custom_emojis_controller.rb

@@ -2,16 +2,16 @@
 
 module Admin
   class CustomEmojisController < BaseController
-    before_action :set_custom_emoji, except: [:index, :new, :create]
-    before_action :set_filter_params
-
     def index
       authorize :custom_emoji, :index?
+
       @custom_emojis = filtered_custom_emojis.eager_load(:local_counterpart).page(params[:page])
+      @form          = Form::CustomEmojiBatch.new
     end
 
     def new
       authorize :custom_emoji, :create?
+
       @custom_emoji = CustomEmoji.new
     end
 
@@ -28,69 +28,19 @@ module Admin
       end
     end
 
-    def update
-      authorize @custom_emoji, :update?
-
-      if @custom_emoji.update(resource_params)
-        log_action :update, @custom_emoji
-        flash[:notice] = I18n.t('admin.custom_emojis.updated_msg')
-      else
-        flash[:alert] =  I18n.t('admin.custom_emojis.update_failed_msg')
-      end
-      redirect_to admin_custom_emojis_path(page: params[:page], **@filter_params)
-    end
-
-    def destroy
-      authorize @custom_emoji, :destroy?
-      @custom_emoji.destroy!
-      log_action :destroy, @custom_emoji
-      flash[:notice] = I18n.t('admin.custom_emojis.destroyed_msg')
-      redirect_to admin_custom_emojis_path(page: params[:page], **@filter_params)
-    end
-
-    def copy
-      authorize @custom_emoji, :copy?
-
-      emoji = CustomEmoji.find_or_initialize_by(domain: nil,
-                                                shortcode: @custom_emoji.shortcode)
-      emoji.image = @custom_emoji.image
-
-      if emoji.save
-        log_action :create, emoji
-        flash[:notice] = I18n.t('admin.custom_emojis.copied_msg')
-      else
-        flash[:alert] = I18n.t('admin.custom_emojis.copy_failed_msg')
-      end
-
-      redirect_to admin_custom_emojis_path(page: params[:page], **@filter_params)
-    end
-
-    def enable
-      authorize @custom_emoji, :enable?
-      @custom_emoji.update!(disabled: false)
-      log_action :enable, @custom_emoji
-      flash[:notice] = I18n.t('admin.custom_emojis.enabled_msg')
-      redirect_to admin_custom_emojis_path(page: params[:page], **@filter_params)
-    end
-
-    def disable
-      authorize @custom_emoji, :disable?
-      @custom_emoji.update!(disabled: true)
-      log_action :disable, @custom_emoji
-      flash[:notice] = I18n.t('admin.custom_emojis.disabled_msg')
-      redirect_to admin_custom_emojis_path(page: params[:page], **@filter_params)
+    def batch
+      @form = Form::CustomEmojiBatch.new(form_custom_emoji_batch_params.merge(current_account: current_account, action: action_from_button))
+      @form.save
+    rescue ActionController::ParameterMissing
+      flash[:alert] = I18n.t('admin.accounts.no_account_selected')
+    rescue Mastodon::NotPermittedError
+      flash[:alert] = I18n.t('admin.custom_emojis.not_permitted')
+    ensure
+      redirect_to admin_custom_emojis_path(filter_params)
     end
 
     private
 
-    def set_custom_emoji
-      @custom_emoji = CustomEmoji.find(params[:id])
-    end
-
-    def set_filter_params
-      @filter_params = filter_params.to_hash.symbolize_keys
-    end
-
     def resource_params
       params.require(:custom_emoji).permit(:shortcode, :image, :visible_in_picker)
     end
@@ -100,12 +50,29 @@ module Admin
     end
 
     def filter_params
-      params.permit(
-        :local,
-        :remote,
-        :by_domain,
-        :shortcode
-      )
+      params.slice(:page, *CustomEmojiFilter::KEYS).permit(:page, *CustomEmojiFilter::KEYS)
+    end
+
+    def action_from_button
+      if params[:update]
+        'update'
+      elsif params[:list]
+        'list'
+      elsif params[:unlist]
+        'unlist'
+      elsif params[:enable]
+        'enable'
+      elsif params[:disable]
+        'disable'
+      elsif params[:copy]
+        'copy'
+      elsif params[:delete]
+        'delete'
+      end
+    end
+
+    def form_custom_emoji_batch_params
+      params.require(:form_custom_emoji_batch).permit(:action, :category_id, :category_name, custom_emoji_ids: [])
     end
   end
 end

app/controllers/admin/dashboard_controller.rb

@@ -5,12 +5,13 @@ module Admin
   class DashboardController < BaseController
     def index
       @users_count           = User.count
+      @pending_users_count   = User.pending.count
       @registrations_week    = Redis.current.get("activity:accounts:local:#{current_week}") || 0
       @logins_week           = Redis.current.pfcount("activity:logins:#{current_week}")
       @interactions_week     = Redis.current.get("activity:interactions:#{current_week}") || 0
       @relay_enabled         = Relay.enabled.exists?
       @single_user_mode      = Rails.configuration.x.single_user_mode
-      @registrations_enabled = Setting.open_registrations
+      @registrations_enabled = Setting.registrations_mode != 'none'
       @deletions_enabled     = Setting.open_deletion
       @invites_enabled       = Setting.min_invite_role == 'user'
       @search_enabled        = Chewy.enabled?
@@ -19,7 +20,7 @@ module Admin
       @redis_version         = redis_info['redis_version']
       @reports_count         = Report.unresolved.count
       @queue_backlog         = Sidekiq::Stats.new.enqueued
-      @recent_users          = User.confirmed.recent.includes(:account).limit(4)
+      @recent_users          = User.confirmed.recent.includes(:account).limit(8)
       @database_size         = ActiveRecord::Base.connection.execute('SELECT pg_database_size(current_database())').first['pg_database_size']
       @redis_size            = redis_info['used_memory']
       @ldap_enabled          = ENV['LDAP_ENABLED'] == 'true'
@@ -27,7 +28,14 @@ module Admin
       @saml_enabled          = ENV['SAML_ENABLED'] == 'true'
       @pam_enabled           = ENV['PAM_ENABLED'] == 'true'
       @hidden_service        = ENV['ALLOW_ACCESS_TO_HIDDEN_SERVICE'] == 'true'
-      @trending_hashtags     = TrendingTags.get(7)
+      @trending_hashtags     = TrendingTags.get(10, filtered: false)
+      @pending_tags_count    = Tag.pending_review.count
+      @authorized_fetch      = authorized_fetch_mode?
+      @whitelist_enabled     = whitelist_mode?
+      @profile_directory     = Setting.profile_directory
+      @timeline_preview      = Setting.timeline_preview
+      @spam_check_enabled    = Setting.spam_check_enabled
+      @trends_enabled        = Setting.trends
     end
 
     private
@@ -37,7 +45,13 @@ module Admin
     end
 
     def redis_info
-      @redis_info ||= Redis.current.info
+      @redis_info ||= begin
+        if Redis.current.is_a?(Redis::Namespace)
+          Redis.current.redis.info
+        else
+          Redis.current.info
+        end
+      end
     end
   end
 end

app/controllers/admin/domain_allows_controller.rb

@@ -0,0 +1,40 @@
+# frozen_string_literal: true
+
+class Admin::DomainAllowsController < Admin::BaseController
+  before_action :set_domain_allow, only: [:destroy]
+
+  def new
+    authorize :domain_allow, :create?
+
+    @domain_allow = DomainAllow.new(domain: params[:_domain])
+  end
+
+  def create
+    authorize :domain_allow, :create?
+
+    @domain_allow = DomainAllow.new(resource_params)
+
+    if @domain_allow.save
+      log_action :create, @domain_allow
+      redirect_to admin_instances_path, notice: I18n.t('admin.domain_allows.created_msg')
+    else
+      render :new
+    end
+  end
+
+  def destroy
+    authorize @domain_allow, :destroy?
+    UnallowDomainService.new.call(@domain_allow)
+    redirect_to admin_instances_path, notice: I18n.t('admin.domain_allows.destroyed_msg')
+  end
+
+  private
+
+  def set_domain_allow
+    @domain_allow = DomainAllow.find(params[:id])
+  end
+
+  def resource_params
+    params.require(:domain_allow).permit(:domain)
+  end
+end

app/controllers/admin/domain_blocks_controller.rb

@@ -2,29 +2,56 @@
 
 module Admin
   class DomainBlocksController < BaseController
-    before_action :set_domain_block, only: [:show, :destroy]
-
-    def index
-      authorize :domain_block, :index?
-      @domain_blocks = DomainBlock.page(params[:page])
-    end
+    before_action :set_domain_block, only: [:show, :destroy, :edit, :update]
 
     def new
       authorize :domain_block, :create?
-      @domain_block = DomainBlock.new
+      @domain_block = DomainBlock.new(domain: params[:_domain])
+    end
+
+    def edit
+      authorize :domain_block, :create?
     end
 
     def create
       authorize :domain_block, :create?
 
       @domain_block = DomainBlock.new(resource_params)
+      existing_domain_block = resource_params[:domain].present? ? DomainBlock.rule_for(resource_params[:domain]) : nil
+
+      if existing_domain_block.present? && !@domain_block.stricter_than?(existing_domain_block)
+        @domain_block.save
+        flash.now[:alert] = I18n.t('admin.domain_blocks.existing_domain_block_html', name: existing_domain_block.domain, unblock_url: admin_domain_block_path(existing_domain_block)).html_safe # rubocop:disable Rails/OutputSafety
+        @domain_block.errors[:domain].clear
+        render :new
+      else
+        if existing_domain_block.present?
+          @domain_block = existing_domain_block
+          @domain_block.update(resource_params)
+        end
+        if @domain_block.save
+          DomainBlockWorker.perform_async(@domain_block.id)
+          log_action :create, @domain_block
+          redirect_to admin_instances_path(limited: '1'), notice: I18n.t('admin.domain_blocks.created_msg')
+        else
+          render :new
+        end
+      end
+    end
+
+    def update
+      authorize :domain_block, :create?
+
+      @domain_block.update(update_params)
+
+      severity_changed = @domain_block.severity_changed?
 
       if @domain_block.save
-        DomainBlockWorker.perform_async(@domain_block.id)
+        DomainBlockWorker.perform_async(@domain_block.id, severity_changed)
         log_action :create, @domain_block
-        redirect_to admin_domain_blocks_path, notice: I18n.t('admin.domain_blocks.created_msg')
+        redirect_to admin_instances_path(limited: '1'), notice: I18n.t('admin.domain_blocks.created_msg')
       else
-        render :new
+        render :edit
       end
     end
 
@@ -34,9 +61,9 @@ module Admin
 
     def destroy
       authorize @domain_block, :destroy?
-      UnblockDomainService.new.call(@domain_block, retroactive_unblock?)
+      UnblockDomainService.new.call(@domain_block)
       log_action :destroy, @domain_block
-      redirect_to admin_domain_blocks_path, notice: I18n.t('admin.domain_blocks.destroyed_msg')
+      redirect_to admin_instances_path(limited: '1'), notice: I18n.t('admin.domain_blocks.destroyed_msg')
     end
 
     private
@@ -45,12 +72,12 @@ module Admin
       @domain_block = DomainBlock.find(params[:id])
     end
 
-    def resource_params
-      params.require(:domain_block).permit(:domain, :severity, :reject_media, :reject_reports, :retroactive)
+    def update_params
+      params.require(:domain_block).permit(:severity, :reject_media, :reject_reports, :private_comment, :public_comment)
     end
 
-    def retroactive_unblock?
-      ActiveRecord::Type.lookup(:boolean).cast(resource_params[:retroactive])
+    def resource_params
+      params.require(:domain_block).permit(:domain, :severity, :reject_media, :reject_reports, :private_comment, :public_comment)
     end
   end
 end

app/controllers/admin/email_domain_blocks_controller.rb

@@ -6,12 +6,12 @@ module Admin
 
     def index
       authorize :email_domain_block, :index?
-      @email_domain_blocks = EmailDomainBlock.page(params[:page])
+      @email_domain_blocks = EmailDomainBlock.where(parent_id: nil).includes(:children).order(id: :desc).page(params[:page])
     end
 
     def new
       authorize :email_domain_block, :create?
-      @email_domain_block = EmailDomainBlock.new
+      @email_domain_block = EmailDomainBlock.new(domain: params[:_domain])
     end
 
     def create
@@ -21,6 +21,28 @@ module Admin
 
       if @email_domain_block.save
         log_action :create, @email_domain_block
+
+        if @email_domain_block.with_dns_records?
+          hostnames = []
+          ips       = []
+
+          Resolv::DNS.open do |dns|
+            dns.timeouts = 1
+
+            hostnames = dns.getresources(@email_domain_block.domain, Resolv::DNS::Resource::IN::MX).to_a.map { |e| e.exchange.to_s }
+
+            ([@email_domain_block.domain] + hostnames).uniq.each do |hostname|
+              ips.concat(dns.getresources(hostname, Resolv::DNS::Resource::IN::A).to_a.map { |e| e.address.to_s })
+              ips.concat(dns.getresources(hostname, Resolv::DNS::Resource::IN::AAAA).to_a.map { |e| e.address.to_s })
+            end
+          end
+
+          (hostnames + ips).each do |hostname|
+            another_email_domain_block = EmailDomainBlock.new(domain: hostname, parent: @email_domain_block)
+            log_action :create, another_email_domain_block if another_email_domain_block.save
+          end
+        end
+
         redirect_to admin_email_domain_blocks_path, notice: I18n.t('admin.email_domain_blocks.created_msg')
       else
         render :new
@@ -41,7 +63,7 @@ module Admin
     end
 
     def resource_params
-      params.require(:email_domain_block).permit(:domain)
+      params.require(:email_domain_block).permit(:domain, :with_dns_records)
     end
   end
 end

app/controllers/admin/instances_controller.rb

@@ -2,22 +2,53 @@
 
 module Admin
   class InstancesController < BaseController
+    before_action :set_domain_block, only: :show
+    before_action :set_domain_allow, only: :show
+    before_action :set_instance, only: :show
+
     def index
       authorize :instance, :index?
+
       @instances = ordered_instances
     end
 
-    def resubscribe
-      authorize :instance, :resubscribe?
-      params.require(:by_domain)
-      Pubsubhubbub::SubscribeWorker.push_bulk(subscribeable_accounts.pluck(:id))
-      redirect_to admin_instances_path
+    def show
+      authorize :instance, :show?
+
+      @following_count = Follow.where(account: Account.where(domain: params[:id])).count
+      @followers_count = Follow.where(target_account: Account.where(domain: params[:id])).count
+      @reports_count   = Report.where(target_account: Account.where(domain: params[:id])).count
+      @blocks_count    = Block.where(target_account: Account.where(domain: params[:id])).count
+      @available       = DeliveryFailureTracker.available?(params[:id])
+      @media_storage   = MediaAttachment.where(account: Account.where(domain: params[:id])).sum(:file_file_size)
+      @private_comment = @domain_block&.private_comment
+      @public_comment  = @domain_block&.public_comment
     end
 
     private
 
+    def set_domain_block
+      @domain_block = DomainBlock.rule_for(params[:id])
+    end
+
+    def set_domain_allow
+      @domain_allow = DomainAllow.rule_for(params[:id])
+    end
+
+    def set_instance
+      resource   = Account.by_domain_accounts.find_by(domain: params[:id])
+      resource ||= @domain_block
+      resource ||= @domain_allow
+
+      if resource
+        @instance = Instance.new(resource)
+      else
+        not_found
+      end
+    end
+
     def filtered_instances
-      InstanceFilter.new(filter_params).results
+      InstanceFilter.new(whitelist_mode? ? { allowed: true } : filter_params).results
     end
 
     def paginated_instances
@@ -27,17 +58,11 @@ module Admin
     helper_method :paginated_instances
 
     def ordered_instances
-      paginated_instances.map { |account| Instance.new(account) }
-    end
-
-    def subscribeable_accounts
-      Account.with_followers.remote.where(domain: params[:by_domain])
+      paginated_instances.map { |resource| Instance.new(resource) }
     end
 
     def filter_params
-      params.permit(
-        :domain_name
-      )
+      params.slice(*InstanceFilter::KEYS).permit(*InstanceFilter::KEYS)
     end
   end
 end

app/controllers/admin/invites_controller.rb

@@ -47,7 +47,7 @@ module Admin
     end
 
     def filter_params
-      params.permit(:available, :expired)
+      params.slice(*InviteFilter::KEYS).permit(*InviteFilter::KEYS)
     end
   end
 end

app/controllers/admin/pending_accounts_controller.rb

@@ -0,0 +1,52 @@
+# frozen_string_literal: true
+
+module Admin
+  class PendingAccountsController < BaseController
+    before_action :set_accounts, only: :index
+
+    def index
+      @form = Form::AccountBatch.new
+    end
+
+    def batch
+      @form = Form::AccountBatch.new(form_account_batch_params.merge(current_account: current_account, action: action_from_button))
+      @form.save
+    rescue ActionController::ParameterMissing
+      flash[:alert] = I18n.t('admin.accounts.no_account_selected')
+    ensure
+      redirect_to admin_pending_accounts_path(current_params)
+    end
+
+    def approve_all
+      Form::AccountBatch.new(current_account: current_account, account_ids: User.pending.pluck(:account_id), action: 'approve').save
+      redirect_to admin_pending_accounts_path(current_params)
+    end
+
+    def reject_all
+      Form::AccountBatch.new(current_account: current_account, account_ids: User.pending.pluck(:account_id), action: 'reject').save
+      redirect_to admin_pending_accounts_path(current_params)
+    end
+
+    private
+
+    def set_accounts
+      @accounts = Account.joins(:user).merge(User.pending.recent).includes(user: :invite_request).page(params[:page])
+    end
+
+    def form_account_batch_params
+      params.require(:form_account_batch).permit(:action, account_ids: [])
+    end
+
+    def action_from_button
+      if params[:approve]
+        'approve'
+      elsif params[:reject]
+        'reject'
+      end
+    end
+
+    def current_params
+      params.slice(:page).permit(:page)
+    end
+  end
+end

app/controllers/admin/relationships_controller.rb

@@ -0,0 +1,25 @@
+# frozen_string_literal: true
+
+module Admin
+  class RelationshipsController < BaseController
+    before_action :set_account
+
+    PER_PAGE = 40
+
+    def index
+      authorize :account, :index?
+
+      @accounts = RelationshipFilter.new(@account, filter_params).results.page(params[:page]).per(PER_PAGE)
+    end
+
+    private
+
+    def set_account
+      @account = Account.find(params[:account_id])
+    end
+
+    def filter_params
+      params.slice(*RelationshipFilter::KEYS).permit(*RelationshipFilter::KEYS)
+    end
+  end
+end

app/controllers/admin/relays_controller.rb

@@ -3,6 +3,7 @@
 module Admin
   class RelaysController < BaseController
     before_action :set_relay, except: [:index, :new, :create]
+    before_action :require_signatures_enabled!, only: [:new, :create, :enable]
 
     def index
       authorize :relay, :update?
@@ -11,7 +12,7 @@ module Admin
 
     def new
       authorize :relay, :update?
-      @relay = Relay.new(inbox_url: Relay::PRESET_RELAY)
+      @relay = Relay.new
     end
 
     def create
@@ -54,5 +55,9 @@ module Admin
     def resource_params
       params.require(:relay).permit(:inbox_url)
     end
+
+    def require_signatures_enabled!
+      redirect_to admin_relays_path, alert: I18n.t('admin.relays.signatures_not_enabled') if authorized_fetch_mode?
+    end
   end
 end

app/controllers/admin/report_notes_controller.rb

@@ -5,10 +5,10 @@ module Admin
     before_action :set_report_note, only: [:destroy]
 
     def create
-      authorize ReportNote, :create?
+      authorize :report_note, :create?
 
       @report_note = current_account.report_notes.new(resource_params)
-      @report = @report_note.report
+      @report      = @report_note.report
 
       if @report_note.save
         if params[:create_and_resolve]
@@ -26,9 +26,8 @@ module Admin
 
         redirect_to admin_report_path(@report), notice: I18n.t('admin.report_notes.created_msg')
       else
-        @report_notes = @report.notes.latest
-        @report_history = @report.history
-        @form = Form::StatusBatch.new
+        @report_notes = (@report.notes.latest + @report.history + @report.target_account.targeted_account_warnings.latest.custom).sort_by(&:created_at)
+        @form         = Form::StatusBatch.new
 
         render template: 'admin/reports/show'
       end

app/controllers/admin/reported_statuses_controller.rb

@@ -10,6 +10,10 @@ module Admin
       @form         = Form::StatusBatch.new(form_status_batch_params.merge(current_account: current_account, action: action_from_button))
       flash[:alert] = I18n.t('admin.statuses.failed_to_execute') unless @form.save
 
+      redirect_to admin_report_path(@report)
+    rescue ActionController::ParameterMissing
+      flash[:alert] = I18n.t('admin.statuses.no_status_selected')
+
       redirect_to admin_report_path(@report)
     end
 

app/controllers/admin/reports_controller.rb

@@ -13,83 +13,46 @@ module Admin
       authorize @report, :show?
 
       @report_note  = @report.notes.new
-      @report_notes = (@report.notes.latest + @report.history).sort_by(&:created_at)
+      @report_notes = (@report.notes.latest + @report.history + @report.target_account.targeted_account_warnings.latest.custom).sort_by(&:created_at)
       @form         = Form::StatusBatch.new
     end
 
-    def update
+    def assign_to_self
       authorize @report, :update?
-      process_report
+      @report.update!(assigned_account_id: current_account.id)
+      log_action :assigned_to_self, @report
+      redirect_to admin_report_path(@report)
+    end
 
-      if @report.action_taken?
-        redirect_to admin_reports_path, notice: I18n.t('admin.reports.resolved_msg')
-      else
-        redirect_to admin_report_path(@report)
-      end
+    def unassign
+      authorize @report, :update?
+      @report.update!(assigned_account_id: nil)
+      log_action :unassigned, @report
+      redirect_to admin_report_path(@report)
+    end
+
+    def reopen
+      authorize @report, :update?
+      @report.unresolve!
+      log_action :reopen, @report
+      redirect_to admin_report_path(@report)
+    end
+
+    def resolve
+      authorize @report, :update?
+      @report.resolve!(current_account)
+      log_action :resolve, @report
+      redirect_to admin_reports_path, notice: I18n.t('admin.reports.resolved_msg')
     end
 
     private
 
-    def process_report
-      case params[:outcome].to_s
-      when 'assign_to_self'
-        @report.update!(assigned_account_id: current_account.id)
-        log_action :assigned_to_self, @report
-      when 'unassign'
-        @report.update!(assigned_account_id: nil)
-        log_action :unassigned, @report
-      when 'reopen'
-        @report.unresolve!
-        log_action :reopen, @report
-      when 'resolve'
-        @report.resolve!(current_account)
-        log_action :resolve, @report
-      when 'disable'
-        @report.resolve!(current_account)
-        @report.target_account.user.disable!
-
-        log_action :resolve, @report
-        log_action :disable, @report.target_account.user
-
-        resolve_all_target_account_reports
-      when 'silence'
-        @report.resolve!(current_account)
-        @report.target_account.update!(silenced: true)
-
-        log_action :resolve, @report
-        log_action :silence, @report.target_account
-
-        resolve_all_target_account_reports
-      else
-        raise ActiveRecord::RecordNotFound
-      end
-
-      @report.reload
-    end
-
-    def resolve_all_target_account_reports
-      unresolved_reports_for_target_account.update_all(action_taken: true, action_taken_by_account_id: current_account.id)
-    end
-
-    def unresolved_reports_for_target_account
-      Report.where(
-        target_account: @report.target_account
-      ).unresolved
-    end
-
     def filtered_reports
-      ReportFilter.new(filter_params).results.order(id: :desc).includes(
-        :account,
-        :target_account
-      )
+      ReportFilter.new(filter_params).results.order(id: :desc).includes(:account, :target_account)
     end
 
     def filter_params
-      params.permit(
-        :account_id,
-        :resolved,
-        :target_account_id
-      )
+      params.slice(*ReportFilter::KEYS).permit(*ReportFilter::KEYS)
     end
 
     def set_report

app/controllers/admin/resets_controller.rb

@@ -10,11 +10,5 @@ module Admin
       log_action :reset_password, @user
       redirect_to admin_accounts_path
     end
-
-    private
-
-    def set_user
-      @user = Account.find(params[:account_id]).user || raise(ActiveRecord::RecordNotFound)
-    end
   end
 end

app/controllers/admin/roles_controller.rb

@@ -17,11 +17,5 @@ module Admin
       log_action :demote, @user
       redirect_to admin_account_path(@user.account_id)
     end
-
-    private
-
-    def set_user
-      @user = Account.find(params[:account_id]).user || raise(ActiveRecord::RecordNotFound)
-    end
   end
 end

app/controllers/admin/settings_controller.rb

@@ -2,83 +2,29 @@
 
 module Admin
   class SettingsController < BaseController
-    ADMIN_SETTINGS = %w(
-      site_contact_username
-      site_contact_email
-      site_title
-      site_short_description
-      site_description
-      site_extended_description
-      site_terms
-      open_registrations
-      closed_registrations_message
-      open_deletion
-      timeline_preview
-      show_staff_badge
-      bootstrap_timeline_accounts
-      theme
-      thumbnail
-      hero
-      mascot
-      min_invite_role
-      activity_api_enabled
-      peers_api_enabled
-      show_known_fediverse_at_about_page
-      preview_sensitive_media
-      custom_css
-    ).freeze
-
-    BOOLEAN_SETTINGS = %w(
-      open_registrations
-      open_deletion
-      timeline_preview
-      show_staff_badge
-      activity_api_enabled
-      peers_api_enabled
-      show_known_fediverse_at_about_page
-      preview_sensitive_media
-    ).freeze
-
-    UPLOAD_SETTINGS = %w(
-      thumbnail
-      hero
-      mascot
-    ).freeze
-
     def edit
       authorize :settings, :show?
+
       @admin_settings = Form::AdminSettings.new
     end
 
     def update
       authorize :settings, :update?
 
-      settings_params.each do |key, value|
-        if UPLOAD_SETTINGS.include?(key)
-          upload = SiteUpload.where(var: key).first_or_initialize(var: key)
-          upload.update(file: value)
-        else
-          setting = Setting.where(var: key).first_or_initialize(var: key)
-          setting.update(value: value_for_update(key, value))
-        end
-      end
+      @admin_settings = Form::AdminSettings.new(settings_params)
 
-      flash[:notice] = I18n.t('generic.changes_saved_msg')
-      redirect_to edit_admin_settings_path
+      if @admin_settings.save
+        flash[:notice] = I18n.t('generic.changes_saved_msg')
+        redirect_to edit_admin_settings_path
+      else
+        render :edit
+      end
     end
 
     private
 
     def settings_params
-      params.require(:form_admin_settings).permit(ADMIN_SETTINGS)
-    end
-
-    def value_for_update(key, value)
-      if BOOLEAN_SETTINGS.include?(key)
-        value == '1'
-      else
-        value
-      end
+      params.require(:form_admin_settings).permit(*Form::AdminSettings::KEYS)
     end
   end
 end

app/controllers/admin/silences_controller.rb

@@ -1,27 +0,0 @@
-# frozen_string_literal: true
-
-module Admin
-  class SilencesController < BaseController
-    before_action :set_account
-
-    def create
-      authorize @account, :silence?
-      @account.update!(silenced: true)
-      log_action :silence, @account
-      redirect_to admin_accounts_path
-    end
-
-    def destroy
-      authorize @account, :unsilence?
-      @account.update!(silenced: false)
-      log_action :unsilence, @account
-      redirect_to admin_accounts_path
-    end
-
-    private
-
-    def set_account
-      @account = Account.find(params[:account_id])
-    end
-  end
-end

app/controllers/admin/site_uploads_controller.rb

@@ -0,0 +1,21 @@
+# frozen_string_literal: true
+
+module Admin
+  class SiteUploadsController < BaseController
+    before_action :set_site_upload
+
+    def destroy
+      authorize :settings, :destroy?
+
+      @site_upload.destroy!
+
+      redirect_to edit_admin_settings_path, notice: I18n.t('admin.site_uploads.destroyed_msg')
+    end
+
+    private
+
+    def set_site_upload
+      @site_upload = SiteUpload.find(params[:id])
+    end
+  end
+end

app/controllers/admin/statuses_controller.rb

@@ -22,6 +22,15 @@ module Admin
       @form     = Form::StatusBatch.new
     end
 
+    def show
+      authorize :status, :index?
+
+      @statuses = @account.statuses.where(id: params[:id])
+      authorize @statuses.first, :show?
+
+      @form = Form::StatusBatch.new
+    end
+
     def create
       authorize :status, :update?
 

app/controllers/admin/suspensions_controller.rb

@@ -1,60 +0,0 @@
-# frozen_string_literal: true
-
-module Admin
-  class SuspensionsController < BaseController
-    before_action :set_account
-
-    def new
-      @suspension = Form::AdminSuspensionConfirmation.new(report_id: params[:report_id])
-    end
-
-    def create
-      authorize @account, :suspend?
-
-      @suspension = Form::AdminSuspensionConfirmation.new(suspension_params)
-
-      if suspension_params[:acct] == @account.acct
-        resolve_report! if suspension_params[:report_id].present?
-        perform_suspend!
-        mark_reports_resolved!
-        redirect_to admin_accounts_path
-      else
-        flash.now[:alert] = I18n.t('admin.suspensions.bad_acct_msg')
-        render :new
-      end
-    end
-
-    def destroy
-      authorize @account, :unsuspend?
-      @account.unsuspend!
-      log_action :unsuspend, @account
-      redirect_to admin_accounts_path
-    end
-
-    private
-
-    def set_account
-      @account = Account.find(params[:account_id])
-    end
-
-    def suspension_params
-      params.require(:form_admin_suspension_confirmation).permit(:acct, :report_id)
-    end
-
-    def resolve_report!
-      report = Report.find(suspension_params[:report_id])
-      report.resolve!(current_account)
-      log_action :resolve, report
-    end
-
-    def perform_suspend!
-      @account.suspend!
-      Admin::SuspensionWorker.perform_async(@account.id)
-      log_action :suspend, @account
-    end
-
-    def mark_reports_resolved!
-      Report.where(target_account: @account).unresolved.update_all(action_taken: true, action_taken_by_account_id: current_account.id)
-    end
-  end
-end

app/controllers/admin/tags_controller.rb

@@ -0,0 +1,103 @@
+# frozen_string_literal: true
+
+module Admin
+  class TagsController < BaseController
+    before_action :set_tag, except: [:index, :batch, :approve_all, :reject_all]
+    before_action :set_usage_by_domain, except: [:index, :batch, :approve_all, :reject_all]
+    before_action :set_counters, except: [:index, :batch, :approve_all, :reject_all]
+
+    def index
+      authorize :tag, :index?
+
+      @tags = filtered_tags.page(params[:page])
+      @form = Form::TagBatch.new
+    end
+
+    def batch
+      @form = Form::TagBatch.new(form_tag_batch_params.merge(current_account: current_account, action: action_from_button))
+      @form.save
+    rescue ActionController::ParameterMissing
+      flash[:alert] = I18n.t('admin.accounts.no_account_selected')
+    ensure
+      redirect_to admin_tags_path(filter_params)
+    end
+
+    def approve_all
+      Form::TagBatch.new(current_account: current_account, tag_ids: Tag.pending_review.pluck(:id), action: 'approve').save
+      redirect_to admin_tags_path(filter_params)
+    end
+
+    def reject_all
+      Form::TagBatch.new(current_account: current_account, tag_ids: Tag.pending_review.pluck(:id), action: 'reject').save
+      redirect_to admin_tags_path(filter_params)
+    end
+
+    def show
+      authorize @tag, :show?
+    end
+
+    def update
+      authorize @tag, :update?
+
+      if @tag.update(tag_params.merge(reviewed_at: Time.now.utc))
+        redirect_to admin_tag_path(@tag.id), notice: I18n.t('admin.tags.updated_msg')
+      else
+        render :show
+      end
+    end
+
+    private
+
+    def set_tag
+      @tag = Tag.find(params[:id])
+    end
+
+    def set_usage_by_domain
+      @usage_by_domain = @tag.statuses
+                             .with_public_visibility
+                             .excluding_silenced_accounts
+                             .where(Status.arel_table[:id].gteq(Mastodon::Snowflake.id_at(Time.now.utc.beginning_of_day)))
+                             .joins(:account)
+                             .group('accounts.domain')
+                             .reorder('statuses_count desc')
+                             .pluck('accounts.domain, count(*) AS statuses_count')
+    end
+
+    def set_counters
+      @accounts_today = @tag.history.first[:accounts]
+      @accounts_week  = Redis.current.pfcount(*current_week_days.map { |day| "activity:tags:#{@tag.id}:#{day}:accounts" })
+    end
+
+    def filtered_tags
+      TagFilter.new(filter_params).results
+    end
+
+    def filter_params
+      params.slice(:page, *TagFilter::KEYS).permit(:page, *TagFilter::KEYS)
+    end
+
+    def tag_params
+      params.require(:tag).permit(:name, :trendable, :usable, :listable)
+    end
+
+    def current_week_days
+      now = Time.now.utc.beginning_of_day.to_date
+
+      (Date.commercial(now.cwyear, now.cweek)..now).map do |date|
+        date.to_time(:utc).beginning_of_day.to_i
+      end
+    end
+
+    def form_tag_batch_params
+      params.require(:form_tag_batch).permit(:action, tag_ids: [])
+    end
+
+    def action_from_button
+      if params[:approve]
+        'approve'
+      elsif params[:reject]
+        'reject'
+      end
+    end
+  end
+end

app/controllers/admin/two_factor_authentications_controller.rb

@@ -2,18 +2,19 @@
 
 module Admin
   class TwoFactorAuthenticationsController < BaseController
-    before_action :set_user
+    before_action :set_target_user
 
     def destroy
       authorize @user, :disable_2fa?
       @user.disable_two_factor!
       log_action :disable_2fa, @user
+      UserMailer.two_factor_disabled(@user).deliver_later!
       redirect_to admin_accounts_path
     end
 
     private
 
-    def set_user
+    def set_target_user
       @user = User.find(params[:user_id])
     end
   end

app/controllers/admin/warning_presets_controller.rb

@@ -0,0 +1,58 @@
+# frozen_string_literal: true
+
+module Admin
+  class WarningPresetsController < BaseController
+    before_action :set_warning_preset, except: [:index, :create]
+
+    def index
+      authorize :account_warning_preset, :index?
+
+      @warning_presets = AccountWarningPreset.alphabetic
+      @warning_preset  = AccountWarningPreset.new
+    end
+
+    def create
+      authorize :account_warning_preset, :create?
+
+      @warning_preset = AccountWarningPreset.new(warning_preset_params)
+
+      if @warning_preset.save
+        redirect_to admin_warning_presets_path
+      else
+        @warning_presets = AccountWarningPreset.alphabetic
+        render :index
+      end
+    end
+
+    def edit
+      authorize @warning_preset, :update?
+    end
+
+    def update
+      authorize @warning_preset, :update?
+
+      if @warning_preset.update(warning_preset_params)
+        redirect_to admin_warning_presets_path
+      else
+        render :edit
+      end
+    end
+
+    def destroy
+      authorize @warning_preset, :destroy?
+
+      @warning_preset.destroy!
+      redirect_to admin_warning_presets_path
+    end
+
+    private
+
+    def set_warning_preset
+      @warning_preset = AccountWarningPreset.find(params[:id])
+    end
+
+    def warning_preset_params
+      params.require(:account_warning_preset).permit(:title, :text)
+    end
+  end
+end

app/controllers/api/base_controller.rb

@@ -7,14 +7,23 @@ class Api::BaseController < ApplicationController
   include RateLimitHeaders
 
   skip_before_action :store_current_location
-  skip_before_action :check_user_permissions
+  skip_before_action :require_functional!, unless: :whitelist_mode?
+
+  before_action :require_authenticated_user!, if: :disallow_unauthenticated_api_access?
+  before_action :set_cache_headers
 
   protect_from_forgery with: :null_session
 
+  skip_around_action :set_locale
+
   rescue_from ActiveRecord::RecordInvalid, Mastodon::ValidationError do |e|
     render json: { error: e.to_s }, status: 422
   end
 
+  rescue_from ActiveRecord::RecordNotUnique do
+    render json: { error: 'Duplicate record' }, status: 422
+  end
+
   rescue_from ActiveRecord::RecordNotFound do
     render json: { error: 'Record not found' }, status: 404
   end
@@ -31,6 +40,18 @@ class Api::BaseController < ApplicationController
     render json: { error: 'This action is not allowed' }, status: 403
   end
 
+  rescue_from Mastodon::RaceConditionError do
+    render json: { error: 'There was a temporary problem serving your request, please try again' }, status: 503
+  end
+
+  rescue_from Mastodon::RateLimitExceededError do
+    render json: { error: I18n.t('errors.429') }, status: 429
+  end
+
+  rescue_from ActionController::ParameterMissing do |e|
+    render json: { error: e.to_s }, status: 400
+  end
+
   def doorkeeper_unauthorized_render_options(error: nil)
     { json: { error: (error.try(:description) || 'Not authorized') } }
   end
@@ -67,13 +88,21 @@ class Api::BaseController < ApplicationController
     nil
   end
 
+  def require_authenticated_user!
+    render json: { error: 'This method requires an authenticated user' }, status: 401 unless current_user
+  end
+
   def require_user!
-    if current_user && !current_user.disabled?
-      set_user_activity
-    elsif current_user
-      render json: { error: 'Your login is currently disabled' }, status: 403
-    else
+    if !current_user
       render json: { error: 'This method requires an authenticated user' }, status: 422
+    elsif current_user.disabled?
+      render json: { error: 'Your login is currently disabled' }, status: 403
+    elsif !current_user.confirmed?
+      render json: { error: 'Your login is missing a confirmed e-mail address' }, status: 403
+    elsif !current_user.approved?
+      render json: { error: 'Your login is currently pending approval' }, status: 403
+    else
+      set_user_activity
     end
   end
 
@@ -84,4 +113,12 @@ class Api::BaseController < ApplicationController
   def authorize_if_got_token!(*scopes)
     doorkeeper_authorize!(*scopes) if doorkeeper_token
   end
+
+  def set_cache_headers
+    response.headers['Cache-Control'] = 'no-cache, no-store, max-age=0, must-revalidate'
+  end
+
+  def disallow_unauthenticated_api_access?
+    authorized_fetch_mode?
+  end
 end

app/controllers/api/oembed_controller.rb

@@ -1,15 +1,25 @@
 # frozen_string_literal: true
 
 class Api::OEmbedController < Api::BaseController
-  respond_to :json
+  skip_before_action :require_authenticated_user!
+
+  before_action :set_status
+  before_action :require_public_status!
 
   def show
-    @status = status_finder.status
     render json: @status, serializer: OEmbedSerializer, width: maxwidth_or_default, height: maxheight_or_default
   end
 
   private
 
+  def set_status
+    @status = status_finder.status
+  end
+
+  def require_public_status!
+    not_found if @status.hidden?
+  end
+
   def status_finder
     StatusFinder.new(params[:url])
   end

app/controllers/api/proofs_controller.rb

@@ -0,0 +1,23 @@
+# frozen_string_literal: true
+
+class Api::ProofsController < Api::BaseController
+  include AccountOwnedConcern
+
+  skip_before_action :require_authenticated_user!
+
+  before_action :set_provider
+
+  def index
+    render json: @account, serializer: @provider.serializer_class
+  end
+
+  private
+
+  def set_provider
+    @provider = ProofProvider.find(params[:provider]) || raise(ActiveRecord::RecordNotFound)
+  end
+
+  def username_param
+    params[:username]
+  end
+end

app/controllers/api/push_controller.rb

@@ -1,73 +0,0 @@
-# frozen_string_literal: true
-
-class Api::PushController < Api::BaseController
-  include SignatureVerification
-
-  def update
-    response, status = process_push_request
-    render plain: response, status: status
-  end
-
-  private
-
-  def process_push_request
-    case hub_mode
-    when 'subscribe'
-      Pubsubhubbub::SubscribeService.new.call(account_from_topic, hub_callback, hub_secret, hub_lease_seconds, verified_domain)
-    when 'unsubscribe'
-      Pubsubhubbub::UnsubscribeService.new.call(account_from_topic, hub_callback)
-    else
-      ["Unknown mode: #{hub_mode}", 422]
-    end
-  end
-
-  def hub_mode
-    params['hub.mode']
-  end
-
-  def hub_topic
-    params['hub.topic']
-  end
-
-  def hub_callback
-    params['hub.callback']
-  end
-
-  def hub_lease_seconds
-    params['hub.lease_seconds']
-  end
-
-  def hub_secret
-    params['hub.secret']
-  end
-
-  def account_from_topic
-    if hub_topic.present? && local_domain? && account_feed_path?
-      Account.find_local(hub_topic_params[:username])
-    end
-  end
-
-  def hub_topic_params
-    @_hub_topic_params ||= Rails.application.routes.recognize_path(hub_topic_uri.path)
-  end
-
-  def hub_topic_uri
-    @_hub_topic_uri ||= Addressable::URI.parse(hub_topic).normalize
-  end
-
-  def local_domain?
-    TagManager.instance.web_domain?(hub_topic_domain)
-  end
-
-  def verified_domain
-    return signed_request_account.domain if signed_request_account
-  end
-
-  def hub_topic_domain
-    hub_topic_uri.host + (hub_topic_uri.port ? ":#{hub_topic_uri.port}" : '')
-  end
-
-  def account_feed_path?
-    hub_topic_params[:controller] == 'accounts' && hub_topic_params[:action] == 'show' && hub_topic_params[:format] == 'atom'
-  end
-end

app/controllers/api/salmon_controller.rb

@@ -1,37 +0,0 @@
-# frozen_string_literal: true
-
-class Api::SalmonController < Api::BaseController
-  include SignatureVerification
-
-  before_action :set_account
-  respond_to :txt
-
-  def update
-    if verify_payload?
-      process_salmon
-      head 202
-    elsif payload.present?
-      render plain: signature_verification_failure_reason, status: 401
-    else
-      head 400
-    end
-  end
-
-  private
-
-  def set_account
-    @account = Account.find(params[:id])
-  end
-
-  def payload
-    @_payload ||= request.body.read
-  end
-
-  def verify_payload?
-    payload.present? && VerifySalmonService.new.call(payload)
-  end
-
-  def process_salmon
-    SalmonWorker.perform_async(@account.id, payload.force_encoding('UTF-8'))
-  end
-end

app/controllers/api/subscriptions_controller.rb

@@ -1,51 +0,0 @@
-# frozen_string_literal: true
-
-class Api::SubscriptionsController < Api::BaseController
-  before_action :set_account
-  respond_to :txt
-
-  def show
-    if subscription.valid?(params['hub.topic'])
-      @account.update(subscription_expires_at: future_expires)
-      render plain: encoded_challenge, status: 200
-    else
-      head 404
-    end
-  end
-
-  def update
-    if subscription.verify(body, request.headers['HTTP_X_HUB_SIGNATURE'])
-      ProcessingWorker.perform_async(@account.id, body.force_encoding('UTF-8'))
-    end
-
-    head 200
-  end
-
-  private
-
-  def subscription
-    @_subscription ||= @account.subscription(
-      api_subscription_url(@account.id)
-    )
-  end
-
-  def body
-    @_body ||= request.body.read
-  end
-
-  def encoded_challenge
-    HTMLEntities.new.encode(params['hub.challenge'])
-  end
-
-  def future_expires
-    Time.now.utc + lease_seconds_or_default
-  end
-
-  def lease_seconds_or_default
-    (params['hub.lease_seconds'] || 1.day).to_i.seconds
-  end
-
-  def set_account
-    @account = Account.find(params[:id])
-  end
-end

app/controllers/api/v1/accounts/credentials_controller.rb

@@ -21,11 +21,11 @@ class Api::V1::Accounts::CredentialsController < Api::BaseController
   private
 
   def account_params
-    params.permit(:display_name, :note, :avatar, :header, :locked, :bot, fields_attributes: [:name, :value])
+    params.permit(:display_name, :note, :avatar, :header, :locked, :bot, :discoverable, fields_attributes: [:name, :value])
   end
 
   def user_settings_params
-    return nil unless params.key?(:source)
+    return nil if params[:source].blank?
 
     source_params = params.require(:source)
 

app/controllers/api/v1/accounts/follower_accounts_controller.rb

@@ -5,8 +5,6 @@ class Api::V1::Accounts::FollowerAccountsController < Api::BaseController
   before_action :set_account
   after_action :insert_pagination_headers
 
-  respond_to :json
-
   def index
     @accounts = load_accounts
     render json: @accounts, each_serializer: REST::AccountSerializer
@@ -19,13 +17,19 @@ class Api::V1::Accounts::FollowerAccountsController < Api::BaseController
   end
 
   def load_accounts
-    return [] if @account.user_hides_network? && current_account.id != @account.id
+    return [] if hide_results?
 
-    default_accounts.merge(paginated_follows).to_a
+    scope = default_accounts
+    scope = scope.where.not(id: current_account.excluded_from_timeline_account_ids) unless current_account.nil? || current_account.id == @account.id
+    scope.merge(paginated_follows).to_a
+  end
+
+  def hide_results?
+    (@account.hides_followers? && current_account&.id != @account.id) || (current_account && @account.blocking?(current_account))
   end
 
   def default_accounts
-    Account.includes(:active_relationships).references(:active_relationships)
+    Account.includes(:active_relationships, :account_stat).references(:active_relationships)
   end
 
   def paginated_follows

app/controllers/api/v1/accounts/following_accounts_controller.rb

@@ -5,8 +5,6 @@ class Api::V1::Accounts::FollowingAccountsController < Api::BaseController
   before_action :set_account
   after_action :insert_pagination_headers
 
-  respond_to :json
-
   def index
     @accounts = load_accounts
     render json: @accounts, each_serializer: REST::AccountSerializer
@@ -19,13 +17,19 @@ class Api::V1::Accounts::FollowingAccountsController < Api::BaseController
   end
 
   def load_accounts
-    return [] if @account.user_hides_network? && current_account.id != @account.id
+    return [] if hide_results?
 
-    default_accounts.merge(paginated_follows).to_a
+    scope = default_accounts
+    scope = scope.where.not(id: current_account.excluded_from_timeline_account_ids) unless current_account.nil? || current_account.id == @account.id
+    scope.merge(paginated_follows).to_a
+  end
+
+  def hide_results?
+    (@account.hides_following? && current_account&.id != @account.id) || (current_account && @account.blocking?(current_account))
   end
 
   def default_accounts
-    Account.includes(:passive_relationships).references(:passive_relationships)
+    Account.includes(:passive_relationships, :account_stat).references(:passive_relationships)
   end
 
   def paginated_follows

app/controllers/api/v1/accounts/identity_proofs_controller.rb

@@ -0,0 +1,17 @@
+# frozen_string_literal: true
+
+class Api::V1::Accounts::IdentityProofsController < Api::BaseController
+  before_action :require_user!
+  before_action :set_account
+
+  def index
+    @proofs = @account.identity_proofs.active
+    render json: @proofs, each_serializer: REST::IdentityProofSerializer
+  end
+
+  private
+
+  def set_account
+    @account = Account.find(params[:account_id])
+  end
+end

app/controllers/api/v1/accounts/lists_controller.rb

@@ -5,8 +5,6 @@ class Api::V1::Accounts::ListsController < Api::BaseController
   before_action :require_user!
   before_action :set_account
 
-  respond_to :json
-
   def index
     @lists = @account.lists.where(account: current_account)
     render json: @lists, each_serializer: REST::ListSerializer

app/controllers/api/v1/accounts/notes_controller.rb

@@ -0,0 +1,30 @@
+# frozen_string_literal: true
+
+class Api::V1::Accounts::NotesController < Api::BaseController
+  include Authorization
+
+  before_action -> { doorkeeper_authorize! :write, :'write:accounts' }
+  before_action :require_user!
+  before_action :set_account
+
+  def create
+    if params[:comment].blank?
+      AccountNote.find_by(account: current_account, target_account: @account)&.destroy
+    else
+      @note = AccountNote.find_or_initialize_by(account: current_account, target_account: @account)
+      @note.comment = params[:comment]
+      @note.save! if @note.changed?
+    end
+    render json: @account, serializer: REST::RelationshipSerializer, relationships: relationships_presenter
+  end
+
+  private
+
+  def set_account
+    @account = Account.find(params[:account_id])
+  end
+
+  def relationships_presenter
+    AccountRelationshipsPresenter.new([@account.id], current_user.account_id)
+  end
+end

app/controllers/api/v1/accounts/pins_controller.rb

@@ -7,8 +7,6 @@ class Api::V1::Accounts::PinsController < Api::BaseController
   before_action :require_user!
   before_action :set_account
 
-  respond_to :json
-
   def create
     AccountPin.create!(account: current_account, target_account: @account)
     render json: @account, serializer: REST::RelationshipSerializer, relationships: relationships_presenter

app/controllers/api/v1/accounts/relationships_controller.rb

@@ -4,8 +4,6 @@ class Api::V1::Accounts::RelationshipsController < Api::BaseController
   before_action -> { doorkeeper_authorize! :read, :'read:follows' }
   before_action :require_user!
 
-  respond_to :json
-
   def index
     accounts = Account.where(id: account_ids).select('id')
     # .where doesn't guarantee that our results are in the same order

app/controllers/api/v1/accounts/search_controller.rb

@@ -4,8 +4,6 @@ class Api::V1::Accounts::SearchController < Api::BaseController
   before_action -> { doorkeeper_authorize! :read, :'read:accounts' }
   before_action :require_user!
 
-  respond_to :json
-
   def show
     @accounts = account_search
     render json: @accounts, each_serializer: REST::AccountSerializer
@@ -16,10 +14,11 @@ class Api::V1::Accounts::SearchController < Api::BaseController
   def account_search
     AccountSearchService.new.call(
       params[:q],
-      limit_param(DEFAULT_ACCOUNTS_LIMIT),
       current_account,
+      limit: limit_param(DEFAULT_ACCOUNTS_LIMIT),
       resolve: truthy_param?(:resolve),
-      following: truthy_param?(:following)
+      following: truthy_param?(:following),
+      offset: params[:offset]
     )
   end
 end

app/controllers/api/v1/accounts/statuses_controller.rb

@@ -1,11 +1,10 @@
 # frozen_string_literal: true
 
 class Api::V1::Accounts::StatusesController < Api::BaseController
-  before_action -> { doorkeeper_authorize! :read, :'read:statuses' }
+  before_action -> { authorize_if_got_token! :read, :'read:statuses' }
   before_action :set_account
-  after_action :insert_pagination_headers
 
-  respond_to :json
+  after_action :insert_pagination_headers, unless: -> { truthy_param?(:pinned) }
 
   def index
     @statuses = load_statuses
@@ -28,15 +27,13 @@ class Api::V1::Accounts::StatusesController < Api::BaseController
 
   def account_statuses
     statuses = truthy_param?(:pinned) ? pinned_scope : permitted_account_statuses
-    statuses = statuses.paginate_by_id(
-      limit_param(DEFAULT_STATUSES_LIMIT),
-      params_slice(:max_id, :since_id, :min_id)
-    )
 
     statuses.merge!(only_media_scope) if truthy_param?(:only_media)
     statuses.merge!(no_replies_scope) if truthy_param?(:exclude_replies)
+    statuses.merge!(no_reblogs_scope) if truthy_param?(:exclude_reblogs)
+    statuses.merge!(hashtag_scope)    if params[:tagged].present?
 
-    statuses
+    statuses.paginate_by_id(limit_param(DEFAULT_STATUSES_LIMIT), params_slice(:max_id, :since_id, :min_id))
   end
 
   def permitted_account_statuses
@@ -52,12 +49,14 @@ class Api::V1::Accounts::StatusesController < Api::BaseController
     # Also, Avoid getting slow by not narrowing down by `statuses.account_id`.
     # When narrowing down by `statuses.account_id`, `index_statuses_20180106` will be used
     # and the table will be joined by `Merge Semi Join`, so the query will be slow.
-    Status.joins(:media_attachments).merge(@account.media_attachments).permitted_for(@account, current_account)
-          .paginate_by_max_id(limit_param(DEFAULT_STATUSES_LIMIT), params[:max_id], params[:since_id])
-          .reorder(id: :desc).distinct(:id).pluck(:id)
+    @account.statuses.joins(:media_attachments).merge(@account.media_attachments).permitted_for(@account, current_account)
+            .paginate_by_max_id(limit_param(DEFAULT_STATUSES_LIMIT), params[:max_id], params[:since_id])
+            .reorder(id: :desc).distinct(:id).pluck(:id)
   end
 
   def pinned_scope
+    return Status.none if @account.blocking?(current_account)
+
     @account.pinned_statuses
   end
 
@@ -65,6 +64,20 @@ class Api::V1::Accounts::StatusesController < Api::BaseController
     Status.without_replies
   end
 
+  def no_reblogs_scope
+    Status.without_reblogs
+  end
+
+  def hashtag_scope
+    tag = Tag.find_normalized(params[:tagged])
+
+    if tag
+      Status.tagged_with(tag.id)
+    else
+      Status.none
+    end
+  end
+
   def pagination_params(core_params)
     params.slice(:limit, :only_media, :exclude_replies).permit(:limit, :only_media, :exclude_replies).merge(core_params)
   end

app/controllers/api/v1/accounts_controller.rb

@@ -1,25 +1,39 @@
 # frozen_string_literal: true
 
 class Api::V1::AccountsController < Api::BaseController
-  before_action -> { authorize_if_got_token! :read, :'read:accounts' }, except: [:follow, :unfollow, :block, :unblock, :mute, :unmute]
+  before_action -> { authorize_if_got_token! :read, :'read:accounts' }, except: [:create, :follow, :unfollow, :block, :unblock, :mute, :unmute]
   before_action -> { doorkeeper_authorize! :follow, :'write:follows' }, only: [:follow, :unfollow]
   before_action -> { doorkeeper_authorize! :follow, :'write:mutes' }, only: [:mute, :unmute]
   before_action -> { doorkeeper_authorize! :follow, :'write:blocks' }, only: [:block, :unblock]
+  before_action -> { doorkeeper_authorize! :write, :'write:accounts' }, only: [:create]
 
-  before_action :require_user!, except: [:show]
-  before_action :set_account
+  before_action :require_user!, except: [:show, :create]
+  before_action :set_account, except: [:create]
   before_action :check_account_suspension, only: [:show]
+  before_action :check_enabled_registrations, only: [:create]
 
-  respond_to :json
+  skip_before_action :require_authenticated_user!, only: :create
+
+  override_rate_limit_headers :follow, family: :follows
 
   def show
     render json: @account, serializer: REST::AccountSerializer
   end
 
-  def follow
-    FollowService.new.call(current_user.account, @account.acct, reblogs: truthy_param?(:reblogs))
+  def create
+    token    = AppSignUpService.new.call(doorkeeper_token.application, account_params)
+    response = Doorkeeper::OAuth::TokenResponse.new(token)
 
-    options = @account.locked? ? {} : { following_map: { @account.id => { reblogs: truthy_param?(:reblogs) } }, requested_map: { @account.id => false } }
+    headers.merge!(response.headers)
+
+    self.response_body = Oj.dump(response.body)
+    self.status        = response.status
+  end
+
+  def follow
+    FollowService.new.call(current_user.account, @account, reblogs: truthy_param?(:reblogs), with_rate_limit: true)
+
+    options = @account.locked? || current_user.account.silenced? ? {} : { following_map: { @account.id => { reblogs: truthy_param?(:reblogs) } }, requested_map: { @account.id => false } }
 
     render json: @account, serializer: REST::RelationshipSerializer, relationships: relationships(options)
   end
@@ -62,4 +76,16 @@ class Api::V1::AccountsController < Api::BaseController
   def check_account_suspension
     gone if @account.suspended?
   end
+
+  def account_params
+    params.permit(:username, :email, :password, :agreement, :locale, :reason)
+  end
+
+  def check_enabled_registrations
+    forbidden if single_user_mode? || !allowed_registrations?
+  end
+
+  def allowed_registrations?
+    Setting.registrations_mode != 'none'
+  end
 end

app/controllers/api/v1/admin/account_actions_controller.rb

@@ -0,0 +1,32 @@
+# frozen_string_literal: true
+
+class Api::V1::Admin::AccountActionsController < Api::BaseController
+  before_action -> { doorkeeper_authorize! :'admin:write', :'admin:write:accounts' }
+  before_action :require_staff!
+  before_action :set_account
+
+  def create
+    account_action                 = Admin::AccountAction.new(resource_params)
+    account_action.target_account  = @account
+    account_action.current_account = current_account
+    account_action.save!
+
+    render_empty
+  end
+
+  private
+
+  def set_account
+    @account = Account.find(params[:account_id])
+  end
+
+  def resource_params
+    params.permit(
+      :type,
+      :report_id,
+      :warning_preset_id,
+      :text,
+      :send_email_notification
+    )
+  end
+end

app/controllers/api/v1/admin/accounts_controller.rb

@@ -0,0 +1,128 @@
+# frozen_string_literal: true
+
+class Api::V1::Admin::AccountsController < Api::BaseController
+  include Authorization
+  include AccountableConcern
+
+  LIMIT = 100
+
+  before_action -> { doorkeeper_authorize! :'admin:read', :'admin:read:accounts' }, only: [:index, :show]
+  before_action -> { doorkeeper_authorize! :'admin:write', :'admin:write:accounts' }, except: [:index, :show]
+  before_action :require_staff!
+  before_action :set_accounts, only: :index
+  before_action :set_account, except: :index
+  before_action :require_local_account!, only: [:enable, :approve, :reject]
+
+  after_action :insert_pagination_headers, only: :index
+
+  FILTER_PARAMS = %i(
+    local
+    remote
+    by_domain
+    active
+    pending
+    disabled
+    silenced
+    suspended
+    username
+    display_name
+    email
+    ip
+    staff
+  ).freeze
+
+  PAGINATION_PARAMS = (%i(limit) + FILTER_PARAMS).freeze
+
+  def index
+    authorize :account, :index?
+    render json: @accounts, each_serializer: REST::Admin::AccountSerializer
+  end
+
+  def show
+    authorize @account, :show?
+    render json: @account, serializer: REST::Admin::AccountSerializer
+  end
+
+  def enable
+    authorize @account.user, :enable?
+    @account.user.enable!
+    log_action :enable, @account.user
+    render json: @account, serializer: REST::Admin::AccountSerializer
+  end
+
+  def approve
+    authorize @account.user, :approve?
+    @account.user.approve!
+    render json: @account, serializer: REST::Admin::AccountSerializer
+  end
+
+  def reject
+    authorize @account.user, :reject?
+    SuspendAccountService.new.call(@account, reserve_email: false, reserve_username: false)
+    render json: @account, serializer: REST::Admin::AccountSerializer
+  end
+
+  def unsilence
+    authorize @account, :unsilence?
+    @account.unsilence!
+    log_action :unsilence, @account
+    render json: @account, serializer: REST::Admin::AccountSerializer
+  end
+
+  def unsuspend
+    authorize @account, :unsuspend?
+    @account.unsuspend!
+    log_action :unsuspend, @account
+    render json: @account, serializer: REST::Admin::AccountSerializer
+  end
+
+  private
+
+  def set_accounts
+    @accounts = filtered_accounts.order(id: :desc).includes(user: [:invite_request, :invite]).paginate_by_id(limit_param(LIMIT), params_slice(:max_id, :since_id, :min_id))
+  end
+
+  def set_account
+    @account = Account.find(params[:id])
+  end
+
+  def filtered_accounts
+    AccountFilter.new(filter_params).results
+  end
+
+  def filter_params
+    params.permit(*FILTER_PARAMS)
+  end
+
+  def insert_pagination_headers
+    set_pagination_headers(next_path, prev_path)
+  end
+
+  def next_path
+    api_v1_admin_accounts_url(pagination_params(max_id: pagination_max_id)) if records_continue?
+  end
+
+  def prev_path
+    api_v1_admin_accounts_url(pagination_params(min_id: pagination_since_id)) unless @accounts.empty?
+  end
+
+  def pagination_max_id
+    @accounts.last.id
+  end
+
+  def pagination_since_id
+    @accounts.first.id
+  end
+
+  def records_continue?
+    @accounts.size == limit_param(LIMIT)
+  end
+
+  def pagination_params(core_params)
+    params.slice(*PAGINATION_PARAMS).permit(*PAGINATION_PARAMS).merge(core_params)
+  end
+
+  def require_local_account!
+    forbidden unless @account.local? && @account.user.present?
+  end
+end

app/controllers/api/v1/admin/reports_controller.rb

@@ -0,0 +1,108 @@
+# frozen_string_literal: true
+
+class Api::V1::Admin::ReportsController < Api::BaseController
+  include Authorization
+  include AccountableConcern
+
+  LIMIT = 100
+
+  before_action -> { doorkeeper_authorize! :'admin:read', :'admin:read:reports' }, only: [:index, :show]
+  before_action -> { doorkeeper_authorize! :'admin:write', :'admin:write:reports' }, except: [:index, :show]
+  before_action :require_staff!
+  before_action :set_reports, only: :index
+  before_action :set_report, except: :index
+
+  after_action :insert_pagination_headers, only: :index
+
+  FILTER_PARAMS = %i(
+    resolved
+    account_id
+    target_account_id
+  ).freeze
+
+  PAGINATION_PARAMS = (%i(limit) + FILTER_PARAMS).freeze
+
+  def index
+    authorize :report, :index?
+    render json: @reports, each_serializer: REST::Admin::ReportSerializer
+  end
+
+  def show
+    authorize @report, :show?
+    render json: @report, serializer: REST::Admin::ReportSerializer
+  end
+
+  def assign_to_self
+    authorize @report, :update?
+    @report.update!(assigned_account_id: current_account.id)
+    log_action :assigned_to_self, @report
+    render json: @report, serializer: REST::Admin::ReportSerializer
+  end
+
+  def unassign
+    authorize @report, :update?
+    @report.update!(assigned_account_id: nil)
+    log_action :unassigned, @report
+    render json: @report, serializer: REST::Admin::ReportSerializer
+  end
+
+  def reopen
+    authorize @report, :update?
+    @report.unresolve!
+    log_action :reopen, @report
+    render json: @report, serializer: REST::Admin::ReportSerializer
+  end
+
+  def resolve
+    authorize @report, :update?
+    @report.resolve!(current_account)
+    log_action :resolve, @report
+    render json: @report, serializer: REST::Admin::ReportSerializer
+  end
+
+  private
+
+  def set_reports
+    @reports = filtered_reports.order(id: :desc).with_accounts.paginate_by_id(limit_param(LIMIT), params_slice(:max_id, :since_id, :min_id))
+  end
+
+  def set_report
+    @report = Report.find(params[:id])
+  end
+
+  def filtered_reports
+    ReportFilter.new(filter_params).results
+  end
+
+  def filter_params
+    params.permit(*FILTER_PARAMS)
+  end
+
+  def insert_pagination_headers
+    set_pagination_headers(next_path, prev_path)
+  end
+
+  def next_path
+    api_v1_admin_reports_url(pagination_params(max_id: pagination_max_id)) if records_continue?
+  end
+
+  def prev_path
+    api_v1_admin_reports_url(pagination_params(min_id: pagination_since_id)) unless @reports.empty?
+  end
+
+  def pagination_max_id
+    @reports.last.id
+  end
+
+  def pagination_since_id
+    @reports.first.id
+  end
+
+  def records_continue?
+    @reports.size == limit_param(LIMIT)
+  end
+
+  def pagination_params(core_params)
+    params.slice(*PAGINATION_PARAMS).permit(*PAGINATION_PARAMS).merge(core_params)
+  end
+end

app/controllers/api/v1/announcements/reactions_controller.rb

@@ -0,0 +1,29 @@
+# frozen_string_literal: true
+
+class Api::V1::Announcements::ReactionsController < Api::BaseController
+  before_action -> { doorkeeper_authorize! :write, :'write:favourites' }
+  before_action :require_user!
+
+  before_action :set_announcement
+  before_action :set_reaction, except: :update
+
+  def update
+    @announcement.announcement_reactions.create!(account: current_account, name: params[:id])
+    render_empty
+  end
+
+  def destroy
+    @reaction.destroy!
+    render_empty
+  end
+
+  private
+
+  def set_reaction
+    @reaction = @announcement.announcement_reactions.where(account: current_account).find_by!(name: params[:id])
+  end
+
+  def set_announcement
+    @announcement = Announcement.published.find(params[:announcement_id])
+  end
+end

app/controllers/api/v1/announcements_controller.rb

@@ -0,0 +1,29 @@
+# frozen_string_literal: true
+
+class Api::V1::AnnouncementsController < Api::BaseController
+  before_action -> { doorkeeper_authorize! :write, :'write:accounts' }, only: :dismiss
+  before_action :require_user!
+  before_action :set_announcements, only: :index
+  before_action :set_announcement, except: :index
+
+  def index
+    render json: @announcements, each_serializer: REST::AnnouncementSerializer
+  end
+
+  def dismiss
+    AnnouncementMute.find_or_create_by!(account: current_account, announcement: @announcement)
+    render_empty
+  end
+
+  private
+
+  def set_announcements
+    @announcements = begin
+      Announcement.published.chronological
+    end
+  end
+
+  def set_announcement
+    @announcement = Announcement.published.find(params[:id])
+  end
+end

app/controllers/api/v1/apps/credentials_controller.rb

@@ -3,9 +3,7 @@
 class Api::V1::Apps::CredentialsController < Api::BaseController
   before_action -> { doorkeeper_authorize! :read }
 
-  respond_to :json
-
   def show
-    render json: doorkeeper_token.application, serializer: REST::StatusSerializer::ApplicationSerializer
+    render json: doorkeeper_token.application, serializer: REST::ApplicationSerializer, fields: %i(name website vapid_key)
   end
 end

app/controllers/api/v1/apps_controller.rb

@@ -1,6 +1,8 @@
 # frozen_string_literal: true
 
 class Api::V1::AppsController < Api::BaseController
+  skip_before_action :require_authenticated_user!
+
   def create
     @app = Doorkeeper::Application.create!(application_options)
     render json: @app, serializer: REST::ApplicationSerializer

app/controllers/api/v1/blocks_controller.rb

@@ -5,8 +5,6 @@ class Api::V1::BlocksController < Api::BaseController
   before_action :require_user!
   after_action :insert_pagination_headers
 
-  respond_to :json
-
   def index
     @accounts = load_accounts
     render json: @accounts, each_serializer: REST::AccountSerializer
@@ -19,7 +17,7 @@ class Api::V1::BlocksController < Api::BaseController
   end
 
   def paginated_blocks
-    @paginated_blocks ||= Block.eager_load(:target_account)
+    @paginated_blocks ||= Block.eager_load(target_account: :account_stat)
                                .where(account: current_account)
                                .paginate_by_max_id(
                                  limit_param(DEFAULT_ACCOUNTS_LIMIT),