Commit Graph

262 Commits

Author SHA1 Message Date
redmatrix 0b012e2995 some prep work 2015-06-29 16:56:18 -07:00
redmatrix 3ecc9efd8d fix for forked thread on diaspora 2015-05-27 16:01:51 -07:00
redmatrix 3fdf1ac458 revert for now 2015-05-27 01:49:47 -07:00
redmatrix 07e8fe1b1c Comments need to have some kind of routing instructions. 2015-05-26 21:00:27 -07:00
friendica 5ac177391b disconnect 2015-04-23 21:13:59 -07:00
friendica 0134a41015 more work on queue optimisations 2015-03-09 19:57:35 -07:00
friendica 2b44c5fc72 Fix a couple of places where we weren't checking for dead hublocs. Add a function to mark a hubloc dead. 2015-02-27 13:43:12 -08:00
friendica ac3384e661 syntax - missing arg 2015-02-25 17:24:09 -08:00
friendica b9e485be54 this may actually fix the deliver loop when deleting existing items. It's hackish but I don't see any other way out. 2015-02-25 16:51:39 -08:00
friendica b32841e2dd Found the cause of the delete looping, and I can prevent it going forward. The issue remains what to do about comments which are already in the DB and have ITEM_ORIGIN incorrectly set. We can't exactly reset them because the "original" context has been lost. (Sorry but couldn't resist an insider pun that none of you will be able to follow anyway). Read the comments. 2015-02-25 15:27:33 -08:00
friendica 1434130264 don't send deleted items upstream - only downstream. 2015-02-24 20:21:21 -08:00
friendica 1807db6cb0 send rating information to directories 2015-02-02 20:13:07 -08:00
friendica 1ef2d1c5c4 updates to put rating info into the local xlink before sending it to known directory servers 2015-01-28 16:52:17 -08:00
friendica aa490a39bb include channel clones in PM deliveries (also some whitespace edits) 2015-01-08 16:02:15 -08:00
friendica b9dc23844c make unique hub filter more robust and straight-forward so it's easier to understand and debug since it's such a critical piece of the delivery chain 2014-12-17 11:57:16 -08:00
friendica eebb6f23fb critical notifier fix to make diaspora delivery work reliably again. If everything breaks revert or debug. 2014-12-17 01:50:43 -08:00
Habeas Codice 1a5a5c7edb PostgreSQL support initial commit
There were 11 main types of changes:
- UPDATE's and DELETE's sometimes had LIMIT 1 at the end of them. This is not only non-compliant but
it would certainly not do what whoever wrote it thought it would. It is likely this mistake was just
copied from Friendica. All of these instances, the LIMIT 1 was simply removed.
- Bitwise operations (and even some non-zero int checks) erroneously rely on MySQL implicit
integer-boolean conversion in the WHERE clauses. This is non-compliant (and bad programming practice
to boot). Proper explicit boolean conversions were added. New queries should use proper conventions.
- MySQL has a different operator for bitwise XOR than postgres. Rather than add yet another dba_
func, I converted them to "& ~" ("AND NOT") when turning off, and "|" ("OR") when turning on. There
were no true toggles (XOR). New queries should refrain from using XOR when not necessary.
- There are several fields which the schema has marked as NOT NULL, but the inserts don't specify
them. The reason this works is because mysql totally ignores the constraint and adds an empty text
default automatically. Again, non-compliant, obviously. In these cases a default of empty text was
added.
- Several statements rely on a non-standard MySQL feature
(http://dev.mysql.com/doc/refman/5.5/en/group-by-handling.html). These queries can all be rewritten
to be standards compliant. Interestingly enough, the newly rewritten standards compliant queries run
a zillion times faster, even on MySQL.
- A couple of function/operator name translations were needed (RAND/RANDOM, GROUP_CONCAT/STRING_AGG,
UTC_NOW, REGEXP/~, ^/#) -- assist functions added in the dba_
- INTERVALs: postgres requires quotes around the value, mysql requires that there are not quotes
around the value -- assist functions added in the dba_
- NULL_DATE's -- Postgres does not allow the invalid date '0000-00-00 00:00:00' (there is no such
thing as year 0 or month 0 or day 0). We use '0001-01-01 00:00:00' for postgres. Conversions are
handled in Zot/item packets automagically by quoting all dates with dbescdate().
- char(##) specifications in the schema creates fields with blank spaces that aren't trimmed in the
code. MySQL apparently treats char(##) as varchar(##), again, non-compliant. Since postgres works
better with text fields anyway, this ball of bugs was simply side-stepped by using 'text' datatype
for all text fields in the postgres schema. varchar was used in a couple of places where it actually
seemed appropriate (size constraint), but without rigorously vetting that all of the PHP code
actually validates data, new bugs might come out from under the rug.
- postgres doesn't store nul bytes and a few other non-printables in text fields, even when quoted.
bytea fields were used when storing binary data (photo.data, attach.data). A new dbescbin() function
was added to handle this transparently.
- postgres does not support LIMIT #,# syntax. All databases support LIMIT # OFFSET # syntax.
Statements were updated to be standard.

These changes require corresponding changes in the coding standards. Please review those before
adding any code going forward.

Still on my TODO list:
- remove quotes from non-reserved identifiers and make reserved identifiers use dba func for quoting
- Rewrite search queries for better results (both MySQL and Postgres)
2014-11-13 12:21:58 -08:00
friendica a659ac8952 disable request message response until we work through this 2014-11-01 00:36:21 -07:00
friendica 28a35261db What this checkin does is catch the case where a comment arrived and there's a missing top-level post to match it with. So we'll send a request back to the sender that you've never seen this thread and please send a fresh copy of the entire conversation to date. We could soon have posts in the matrix from different platforms from days gone by, which have been migrated into the modern world. We'll be polite and not deliver these to everybody. However, if someone comments on one of these antique threads we wouldn't be able to see it in our own matrix because we won't have a copy of the parent post. So this rectifies that situation. Be aware that item deletion may need to change to keep "hard deleted" items indefinitely so that they don't keep coming back. We'll have to null out the important data of the former item to accomplish the deletion aspect. 2014-10-30 19:15:03 -07:00
friendica ad86cc24c8 d* PM cont. 2014-09-23 04:34:35 -07:00
friendica b735961560 found the sucker 2014-09-22 16:23:59 -07:00
friendica e76ab30467 we don't always set HUBLOC_FLAGS_DELETED but we often set hubloc_status = HUBLOC_OFFLINE instead. Filter this as well from deliveries. 2014-09-21 15:11:47 -07:00
friendica 5ea6f27792 final piece of the mrjive delivery puzzle, and now we've got most all the tools to implement mod_hubman 2014-09-15 13:37:57 -07:00
friendica 0dc33900b6 provide a way to sync locations and get rid of bogus hublocs, now implemented 2014-09-14 22:19:19 -07:00
friendica b1809f5f0c log why this doesn't get executed 2014-09-12 15:10:53 -07:00
friendica 6a42d6fe3b works one way sort of - now to go the other way - which will require fixing a whole slew of FIXMEs 2014-08-26 18:58:48 -07:00
friendica e42205cec2 houston we have liftoff 2014-08-26 17:32:51 -07:00
friendica 6e99848043 more diaspora stuff - most of the basic bits are there except queueing and physical delivery (I'm keeping delivery turned off until some of the bugs are fixed so we don't cause inifinite loops or network meltdowns) - now it's just a matter of going through and methodically finding all the bugs 2014-08-25 22:43:44 -07:00
friendica eb8f194419 sort out some anomalies 2014-08-25 21:42:46 -07:00
friendica 84ff380302 more diaspora heavy lifting 2014-08-25 20:57:10 -07:00
friendica 9196c9eef0 We really can't do this without a hubloc. I was hoping we could, but notifier is setup to take hublocs, not xchans. 2014-08-22 21:37:08 -07:00
friendica 5f41813845 homepage revisions 2014-07-06 19:15:09 -07:00
friendica 62b4ec4e75 make folks from certain other networks feel more at home here 2014-05-02 17:29:04 -07:00
friendica 1eb8e570bf tag anyone 2014-04-15 18:35:22 -07:00
friendica 7e6804b599 make sure we can find that function 2014-04-11 15:14:01 -07:00
friendica 202582cbd6 we also need a loop breaker 2014-04-11 15:08:40 -07:00
friendica 50a7e9ce8f this should improve queue performance dramatically. 2014-03-24 19:40:36 -07:00
friendica caa284e786 we don't allow forgeries to be received - save some bandwidth by not allowing them to be sent. We do have to figure out how some secondary delivery chains are ending up with posts belonging to the original owner instead of the chain owner, but that's for another day. 2014-03-21 01:14:51 -07:00
friendica 66baa3cab0 try to reduce the number of simulaneous deliveries of the same post when dealing with owner relays that have more than one channel instance. If things melt down in the next few hours and I'm not available please revert this. I've reviewed a few times and think it's OK, but this part of the delivery code is traditionally touchy. 2014-01-22 22:07:47 -08:00
friendica be8a7e2de6 this is necessary for any possibility of a federated future. 2013-11-21 15:58:42 -08:00
friendica 4791b2fd9c add aid to notifiy table which we may need to supress duplicate notify emails across your channels
also try to handle the wretched mess of broken and duplicated hublocs that fred.cepheus.uberspace.de typically reports
2013-11-20 21:09:13 -08:00
friendica 0b0bd3c207 create update record for accounts on directory server itself during profile changes, re-queue failed directory sync packets, fix account_id index in queue creation, be more selective about updating hubloc_connected (only when we're talking to the associated site). But we still need a way to flag dead accounts and these mechanisms won't cut it, because it isn't a requirement that channels communicate with anybody, except for contacting the directory when the account is created. I think we need to make every channel ping the directory once a month. Then we can find those that have not done so (after syncing with other directories). We can't have the directory just flag dead accounts as this would create a way for an anti-social attacker to become a directory server and mark all the accounts dead. 2013-10-24 19:38:50 -07:00
friendica ad29d0f9a1 preserve the source owner when creating a delivery fork so that we can uplink back to them without any ambiguity. 2013-10-13 18:36:28 -07:00
friendica ff2ada207a Pieces we'll need to tie together chanman and account/channel deletion and directory sync. Please do not mess with any of this.
Please do not mess with any of this. Please do not mess with any of this. Please do not mess with any of this. Please do not mess with any of this. Please do not mess with any of this. Please do not mess with any of this. Please do not mess with any of this. Please do not mess with any of this. Please do not mess with any of this. Please do not mess with any of this. Please do not mess with any of this. Please do not mess with any of this. Please do not mess with any of this. Please do not mess with any of this. Please do not mess with any of this. Please do not mess with any of this. Please do not mess with any of this. Please do not mess with any of this. Please do not mess with any of this. Please do not mess with any of this. Please do not mess with any of this. Please do not mess with any of this. Please do not mess with any of this. Please do not mess with any of this.

OK? Understood?
2013-09-17 20:50:09 -07:00
friendica 83e086edcd doc update, put more telemetry on notifier and try to ensure that private posts have recipients. 2013-09-13 04:51:55 -07:00
friendica a09829a1ee perform a refresh_all when an import has completed so that all your connections get a fresh update. 2013-09-08 21:05:39 -07:00
friendica a35d440ff1 Add Comanche layouts to the things that don't go offsite. 2013-09-01 20:11:04 -07:00
friendica ac541a7d32 most of expire is now ported from friendica, but the protocol bits to push out the delete notifications for the entire batch to all recipients of the original post are not complete and will take quite a bit more work. As a consequence, expire has been completely disabled until it is fully implmented since it could result in completely un-removable posts reminiscent of the infamous "Bonnie Nadri" incident at Diaspora which we do not wish to re-live. 2013-08-14 20:09:02 -07:00
friendica 04f8de184e block content-type (called ITEM_BUILDBLOCK to avoid confusion with content that is blocked) 2013-08-12 17:42:52 -07:00
friendica 85c7d7165f yhis should fix the privacy leak - as well as non-private replies to private posts 2013-08-01 14:27:40 -07:00